3
$\begingroup$

Given the hierarchical model $$ \begin{align} k & \sim \text{Categorical}(\pi_1, \dots, \pi_K) \\ X \mid k & \sim \text{Bernoulli}(\theta_{k}) \end{align} $$ and an i.i.d. sample $X_1, \dots, X_N$, it is standard to use the EM algorithm to produce a joint estimate of $(\boldsymbol{\theta}, \boldsymbol{\pi})$.

But what if the mixture proportions $\pi_1, \dots, \pi_K$ are known? In this case, is there a closed form maximizer for the likelihood of the unknown Bernoulli parameters $\theta_1, \dots, \theta_K$, $$ \mathcal{L}(\boldsymbol{\theta}) = \prod_{n = 1}^N \sum_{k=1}^K \pi_k \theta_k ^{X_n} (1 - \theta_k) ^{1 - X_n} $$

$\endgroup$

1 Answer 1

2
$\begingroup$

No, the parameters are not identifiable. For example, if $\boldsymbol{\pi} = (.2, .3, .5)$ and the samples are evenly split between 0's and 1's, $\boldsymbol{\theta}$ could be (0, 0, 1), (1, 1, 0), (.5, .5, .5), (.25, .25, .75), etc.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.