Suppose data $X=\{x_1,...,x_n\}$ is a mix (union) of two groups $Y_1=\{y_1^{(1)},...,y_{n_1}^{(1)}\}$ and $Y_2=\{y_1^{(2)},...,y_{n_2}^{(2)}\}$, and $Y_1,Y_2$ are generated by two different beta distributions $\text{beta}_1(a_1,b_1)$ and $\text{beta}_2(a_2,b_2)$. The task is to infer which data entry belongs to which group, i.e. to get "labels" $l_i=1$ or $l_i=2$ for each $x_1,...,x_n$

  1. Y_1 is much larger than $Y_2$, say $n_1=10n_2$
  2. the mean of $\text{beta}_1$ is assumed to be smaller than $\text{beta}_2$, say $\frac{a_1}{a_1+b_1} =0.4,\frac{a_2}{a_2+b_2} =0.9$.

First, make an initial guess of $l_i$, e.g. $l_i$ = 2 if $x_i>0.6$, and otherwise $l_i=1$; so we have two "guessed" groups. The initial guess of $a_1,b_1,a_2,b_2$ is by setting $b_1,b_2=1$ and $a_1,a_2$ such that $\frac{a_1}{a_1+b_1},\frac{a_2}{a_2+b_2}$ are the average of each "guessed" group. Note the initial guessed group will be unbalanced as well.

Then update formulas for the parameters are based on log-likelihood

$\ln L=\sum\limits_{i = 1}^2 {(({a_i} - 1)\sum\limits_{j = 1}^{{n_i}} {\ln y_j^{(i)}} + ({b_i} - 1)\sum\limits_{j = 1}^{{n_i}} {\ln (1 - y_j^{(i)})} + {n_i}(\ln \Gamma ({a_i} + {b_i}) - \ln \Gamma ({a_i}) - \ln \Gamma ({b_i})))} $

The inference formulas are thus

${a_i} = {\psi ^{ - 1}}(\psi ({a_i} + {b_i}) + \frac{{\sum\limits_{j = 1}^{{n_i}} {\ln y_j^{(i)}} }}{{{n_i}}})$

${b_i} = {\psi ^{ - 1}}(\psi ({a_i} + {b_i}) + \frac{{\sum\limits_{j = 1}^{{n_i}} {\ln (1 - y_j^{(i)})} }}{{{n_i}}})$

${l_i} = \arg {\max _{{l_i} = 1,2}}\{ \ln L\} $

The problem is: the first iteration will already give high concentration $a_1+b_1$ to the larger group $Y_1$, because it contains much more data, so model seems to be "confident" in the mean of $Y_1$. The concentration $a_2+b_2$ for the smaller group will be like 3 to five times smaller.

As a result, the iteration drives "marginal" data in the first group that is a bit far away from the mean to second group, and then becomes even more concentrated, and the second group becomes even less concentrated. It continues until the first group drives almost all its data to the second group!


What should I do in this situation? Is there any known technique for above problem, to infer group label for data from a mixture of beta distribution, preventing the high concentration of the larger group?

  • $\begingroup$ I didn't trt it, but my intuition tells me that adding a parameter for the probability to be in each group might help. Thus, the estimate for $ l_i $ will be weighted argmax instead of balanced argmax. $\endgroup$ – tmrlvi Jan 27 '18 at 3:51
  • $\begingroup$ Why don't you use EM? $\endgroup$ – Xi'an Jan 27 '18 at 9:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.