0
$\begingroup$

I'm working with experimental data. Participants were assigned to one of three conditions and made a single choice (A, B, C, or D).

So the data look something like this, where the values are counts (made up numbers because I don't have them in front of me):

Condition 1: 35 A's, 23 B's, 17 C's, 2 D's

Condition 2: 17 A's, 73 B's, 45 C's, 23 D's

Condition 3: 71 A's, 15 B's, 61 C's, 17 D's

A possibility I would like to rule out is that some portion, p, of the participants in Condition 1 treat it the same way they would condition 2, while the rest, 1-p, treat it the same way they would condition 3, thus making the distribution of choices in condition 1 just a mixture of the distribution of the choices in the other two.

What I would like to do is:

1) Find p* that maximizes the likelihood of some mixture of the distribution in Condition 2 and that in Condition 3 producing the observed choices in Condition 1.

2) A goodness of fit test to show (hopefully) that even with the best fitting p*, the distribution observed in Condition 1 is unlikely to be a mixture of that in the other conditions.

I have found some similar problems solved online, BUT they all involved THEORETICAL distributions as the constituent elements of the mixture. Is there a way to do this given that I only have empirical distributions?

I am considering the following approach (which I'm guessing is probably wrong):

Because I only care about the estimate p*, not its distributional characteristics, and because the sample proportions are the MLEs for the multinomial probabilities, I will ignore the uncertainty in the sample proportions for the first step. That is, I will calculate the likelihood of a given value of p as:

(p*(17 / (17 + 73 + 45 + 23)) + (1-p)*(71 / (71 + 15 + 61 + 17))^35 + ... (for each choice option A through D)

and maximize to obtain p*.

Then for the next step I will calculate a goodness-of-fit measure for the hypothesis that my sample proportions in condition 1 and the proportions from the mixture model come from the same distribution. This will serve as the test statistic, but I cannot use standard critical values because I have ignored the uncertainty in the same proportions. Rather I would use permutation to simulate the distribution of that same test statistic under the null.

Thoughts?

Improve this question
$\endgroup$
4
  • $\begingroup$ I take it you want to incorporate the fact that the second and third conditions are themselves only observations on some presumed underlying distribution. So under the null hypothesis there's some population multinomial underlying conditions 2 and 3 and the multinomial for 1 is itself a mixture of them? $\endgroup$
    – Glen_b
    Oct 11, 2016 at 23:54
  • $\begingroup$ Yes, that is exactly right. $\endgroup$
    – user2726768
    Oct 12, 2016 at 17:49
  • $\begingroup$ Your updated proposal is incomplete. If we call the sample proportion vectors $P_1,P_2,P_3$, then you have given a formula for $\hat{P}_1[p]=pP_2+(1-p)P_3$, i.e. a prediction for $P_1$ as a function of $p$. However this is not a likelihood. If you mean "minimize $\|P_1-\hat{P}_1[p]\|$", then this would correspond to a likelihood under a model where the entries of $P_1$ are i.i.d. normal (implied by the squared loss function). Whatever loss function you optimize, it will correspond to some implicit model for "the uncertainty in the sample proportions". $\endgroup$
    – GeoMatt22
    Oct 12, 2016 at 21:54
  • $\begingroup$ I'm basically trying to follow the procedure outlined here: ocw.mit.edu/courses/mathematics/… but rather than knowing the true component distributions I only have samples from them. $\endgroup$
    – user2726768
    Oct 13, 2016 at 17:46

1 Answer 1

Reset to default
1
$\begingroup$

If I understand correctly, the "theoretical" name for your empirical distributions is actually multinomial.

Now it is unlikely that your counts for condition 1 are an exact weighted average (convex combination) of your sample counts for conditions 2 and 3. However your problem is to estimate the likelihood that there is a consistent set of underlying multinomial probabilities, consistent with your sample proportions, where condition 1 is an exact mixture of conditions 2 and 3.

The maximum likelihood estimates of your probabilities are the sample proportions, but these have some uncertainty, due to the finite sample size*. You could perhaps proceed using uncertainty estimates such as discussed here (which recommends this paper).

(*There may also be uncertainty in whether or not the "multinomial trials", i.e. participant responses, can be treated as i.i.d. variables as well.)

Improve this answer
$\endgroup$
1
  • $\begingroup$ I am considering the following approach (which I'm guessing is probably wrong): $\endgroup$
    – user2726768
    Oct 12, 2016 at 18:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.