Suppose that we have a mixture model:

$$ p_\theta(y) = \sum_{k = 1}^{K}w_k \phi(y;\mu_k, \sigma^2_kI_d) $$

where $\phi(y;\mu_k, \sigma^2_k)$ is the normal density at $y$ with mean vector $\mu$ and d-dimensional variance matrix $\sigma^2I_d$. $\theta$ contains the weights, means, and variances. I am wondering what generally happens if $K$, the number of groups, is very large compared to $d$, the number of dimensions. For example, if $K = 100$ and $d=2$ or $d=3$. Would this cause identifiability issues?

  • 1
    $\begingroup$ No it is unrelated. The size of each observation is $d$ as well, so there is no identifiability per se, apart from invariance by permutation of the indices. $\endgroup$ – Xi'an Jun 7 '18 at 11:26
  • $\begingroup$ What sort of identifiability issues are you worried about? $\endgroup$ – HStamper Jun 7 '18 at 19:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.