0
$\begingroup$

How many parameters do we need to learn for a k-Gaussian mixture model, where all mixtures have the same spherical radius?

$\endgroup$
1
$\begingroup$

In general a mixture model with $M$ components is defined by

$$f(x)=\sum_{m=1}^{M} \alpha_m \phi(x;\mu_m;\Sigma_m)$$

with $M$ the number of components in the mixture, $\alpha_m$ the mixture weight of the $m$-th component and $\phi(x;\mu_m;\Sigma_m)$ being the Gaussian density function with mean $\mu_m$ and covariance matrix $\Sigma_m$.

As such we usually have the parameters of $M$ uni-/multivariate normal distributions to learn, those are the mean and (co-)variance of each component. For the univariate case this gives us $2\times M$ parameters for the components, if the variance is known it reduces to $M$.
As @gunes correctly noted in his comment, in a multivariate setting with $D$ dimensions it requires $D$ values in the mean vector and $D(D+1)/2$ entries in the covariance matrix.

On top of this the $M$ mixture weights $\alpha_1 \cdots \alpha_M$ need to be determined. Since these sum to one, it is sufficient to determine $M-1$ weights.

Generally:

  • Univariate: $2\times M + M-1$ parameters
  • Multivariate: $MD + M(D(D+1)/2) + M-1$ parameters

Known variance:

  • Univariate: $2M-1$ parameters
  • Multivariate: $MD+M-1$ parameters
$\endgroup$
2
  • 2
    $\begingroup$ A few additions to your answer: Knowing $M-1$ weights is enough, and the general case should consider the dimension for number of parameters, i.e. for the covariance we need $n(n+1)/2$ terms, and for the mean we need $n$ terms. $\endgroup$ – gunes Nov 15 '19 at 7:04
  • $\begingroup$ Good point, I'll edit the answer accordingly $\endgroup$ – deemel Nov 15 '19 at 7:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.