Difference in 2 groups when group assignment is not certain

Suppose you have two groups and you want to see whether these two groups differ in regards to some variable. This sounds like a basic t-test or perhaps non-parametric Wilcoxon rank sum test.

Suppose that the group membership is fuzzy. For example, for each sample, you have some probability that it belongs to group 1 (e.g. $p_1$) and some probability it belongs to group 2 ($p_2=1-p_1$). What are some approaches you can take to determine whether these two groups differ in their outcome variable (say $Y$).

First thing I thought about was a linear mixed effects model. What are some other approaches to this? Pros and cons?

• Just to clarify, I assume the probability of group membership depends upon which group it's in? – jbowman Nov 12 '14 at 20:57
• As in is the data missing at random or is there some dependence? Not sure; This problem was brought up to me in passing, and I haven't sat down and gone over the intricacies of the data and how they arise yet. – bdeonovic Nov 12 '14 at 21:37
• Do you know $p_1,p_2$? Are they constant for each sample? What exactly is the form of your data? – user44764 Nov 13 '14 at 0:37
• Each sample has its own $p_1$ and $p_2$, suppose they are known. – bdeonovic Nov 13 '14 at 3:26

We have data in the form $\{y_i, p_i\}_{i=1,\dots,n}$ where $p_i$ is the probability that $y_i \sim F_1$ and $1-p_i$ is the probability that $y_i \sim F_2$. We introduce a latent parameter $\lambda_i$ such that \begin{align} y_i \mid \lambda_i = k &\sim F_k, \end{align} which is equivalent to the model specification we started with when \begin{align} \mathbb{P}(\lambda_i = 1) &= p_i\\ \mathbb{P}(\lambda_i = 2) &= 1- p_i \end{align}

Observe that if we know $\boldsymbol\lambda$, then we know which $\mathbf{y}$ came from $F_k$. Denote these as $\mathbf{y}^{(k)}$. Suppose also we can calculate the posterior of $F_k \mid \mathbf{y}^{(k)}$ through some method (e.g., conjugacy).

Using total probability, we can write \begin{align} \pi(F_k \mid \mathbf{y}) &= \mathbb{E}_{\boldsymbol\lambda}[\pi(F_k \mid \boldsymbol \lambda, \mathbf{y})]\\ &= \mathbb{E}_{\boldsymbol\lambda}[\pi(F_k \mid \mathbf{y}^{(k)})]\\ &= \sum_{\boldsymbol \lambda \in \boldsymbol \Lambda} \mathbb{P}(\boldsymbol \lambda) \pi(F_k \mid \mathbf{y}^{(k)}), \end{align} where we write $\boldsymbol \Lambda$ to denote the set of all binary $n$-tuples $\boldsymbol \lambda$ can take on (e.g., $\lambda_1 = 0, \lambda_2 = 1,\dots$). This result is enough to get us what we need, since $$\mathbb{P}(\boldsymbol \lambda) = \prod_{i=1}^n p_i^{I\{\lambda_i = 1\}} (1-p_i)^{I\{\lambda_i = 2\}}.$$

In all, this gives us $$\pi(F_k \mid \mathbf{y}) = \sum_{\boldsymbol \lambda \in \boldsymbol \Lambda}\left[ \left( \prod_{i=1}^n p_i^{I\{\lambda_i = 1\}} (1-p_i)^{I\{\lambda_i = 2\}} \right) \pi(F_k \mid \mathbf{y}^{(k)}) \right].$$ Using Monte Carlo, inference isn't so bad. For example,

1. Assign $\lambda_i$ at random according to probability $p_i$
2. Group each $y_i$ based on the simulation of $\lambda_i$ into $\mathbf{y}^{(k=1,2)}$
3. Draw from the posterior of $\pi(F_k \mid \mathbf{y}^{(k)})$ for each $k=1,2$.
4. Repeat

Then we can use the posterior draws obtained in step 3 as approximate draws from the posterior $\pi(F_k \mid \mathbf{y})$. These posterior draws should be able to answer the typical questions of interest..

• It looks like you may have mixed up the sum and the product. If the data are independent, the final expression for the likelihood ought to be a product. After all, isn't this just a mixture model with known mixture probabilities? Thus the likelihood of each $y_i$ should equal $p_i F_1(y_i) + (1-p_i) F_2(y_i)$ and the full likelihood is the product of these sums. I'm guessing, though, because I haven't figured out what much of your notation means. – whuber Nov 13 '14 at 4:27
• The likelihoods are identical; one can see this by distributing and then rearranging the terms of the likelihood you're describing. I introduce the latent variable as computational strategy for inference. I will make some adjustments to the notation to make this clearer. – user44764 Nov 13 '14 at 4:33
• This seems like a reasonable approach and close to the lines of what I was thinking. – bdeonovic Nov 26 '14 at 19:30