# Number of Parameters to be learned in k Guassian Mixture model?

How many parameters do we need to learn for a k-Gaussian mixture model, where all mixtures have the same spherical radius?

In general a mixture model with $$M$$ components is defined by

$$f(x)=\sum_{m=1}^{M} \alpha_m \phi(x;\mu_m;\Sigma_m)$$

with $$M$$ the number of components in the mixture, $$\alpha_m$$ the mixture weight of the $$m$$-th component and $$\phi(x;\mu_m;\Sigma_m)$$ being the Gaussian density function with mean $$\mu_m$$ and covariance matrix $$\Sigma_m$$.

As such we usually have the parameters of $$M$$ uni-/multivariate normal distributions to learn, those are the mean and (co-)variance of each component. For the univariate case this gives us $$2\times M$$ parameters for the components, if the variance is known it reduces to $$M$$.
As @gunes correctly noted in his comment, in a multivariate setting with $$D$$ dimensions it requires $$D$$ values in the mean vector and $$D(D+1)/2$$ entries in the covariance matrix.

On top of this the $$M$$ mixture weights $$\alpha_1 \cdots \alpha_M$$ need to be determined. Since these sum to one, it is sufficient to determine $$M-1$$ weights.

Generally:

• Univariate: $$2\times M + M-1$$ parameters
• Multivariate: $$MD + M(D(D+1)/2) + M-1$$ parameters

Known variance:

• Univariate: $$2M-1$$ parameters
• Multivariate: $$MD+M-1$$ parameters
• A few additions to your answer: Knowing $M-1$ weights is enough, and the general case should consider the dimension for number of parameters, i.e. for the covariance we need $n(n+1)/2$ terms, and for the mean we need $n$ terms. – gunes Nov 15 '19 at 7:04
• Good point, I'll edit the answer accordingly – deemel Nov 15 '19 at 7:10