doubt regarding the gaussian mixture model definition

With reference to the following definition of GMM (see snapshot from Reynolds (1)),

I have two doubts:

In the definition of probability density, the covariance matrix (denoted by sigma) is represented as a vector (since it is represented in bold letters) for each component density. How is it possible? In my opinion, there must be a single covariance matrix (as a scalar and not as a vector) for each component densities.

Secondly, is my interpretation correct when I say that the mean vector (denoted by $mu$) will also be $D$ dimensional if the the data vector $x$ is $D$ dimensional for each component density?

I also think that the order of covariance matrix must be $D\times D$. Correct me if I am wrong.

(1): Douglas Reynolds,
"Gaussian Mixture Models"
Tutorial paper
MIT Lincoln Laboratory
Lexington, MA

• Could you provide a link or a reference from where you have this definition? – Nikolas Rieble Sep 23 '16 at 9:46
• I was reading it from a tutorial paper that I was reading (I may not be able to provide you with the url...However here are the details: TOPIC-Gaussian Mixture Models; AUTHOR- Douglas Reynolds; UNIVERSITY- MIT Lincoln Laboratory, 244 Wood St., Lexington, MA 02140, USA) – Upendra Pratap Singh Sep 23 '16 at 9:50
• Do you have a year for the document it's taken from? – Glen_b Sep 23 '16 at 9:55

is represented as a vector (since it is represented in bold letters)

Not so. It's bold, but it's a capital letter, which usually represents a matrix, and it has to be a matrix to be a covariance. There's nothing that forces a bold letter to be a vector rather than a matrix.

Each of the components are themselves multivariate Gaussian, so each has a covariance matrix of dimension $D\times D$. There are $M$ such components.

is my interpretation correct when I say that the mean vector (denoted by $\mu$) will also be $D$ dimensional if the the data vector $x$ is $D$ dimensional for each component density?

Correct -- and consequently the variance-covariance matrix of the $x$'s ($\Sigma_i$) must be $D\times D$ as stated above.

I also think that the order of covariance matrix must be DxD. Correct me if I am wrong.

Correct -- and so it is. But earlier you said:

In my opinion, there must be a single covariance matrix (as a scalar

so you have contradicted yourself there. It can't be a scalar; it's a matrix as you state at the end.

• A capital represents a matrix, true and bold represents a vector. Therefore it is vector of matrices. Or to speak in code language: It is an array which contains covariance matrix for distribution i on position i. – Nikolas Rieble Sep 23 '16 at 9:46
• @NikolasRieble -- That's not how it works. – Glen_b Sep 23 '16 at 9:47
• You reputation almost conviced me itself. I understand now that mathematically speaking, Sigma is only a covariance matrix and not a vector at all. Yet when implementing it using code, I computed an array of covariance matrices where I used to i-th element for the i-th distribution. This question reminded me of my code, so i mixed up mathematical and code language. You are (as you know) right. – Nikolas Rieble Sep 23 '16 at 9:48

You are right. The bold sigma is a vector of covariance matrices. Yet not the vector is used, but the i-th entry. The same counts for mu - which is a vector of all centers. Yet only the i-th entry is used for the i-th distribution.

Therefore the formula is just fine.

EDIT: Sigma is not a vector, but only a covariance matrix. Implementing algorithms such as EM for GMM (Expectation Maximizization for Gaussian Mixture Models) one tends to (I did) save the covariance matrices in an array and uses the i-th element for the i-th distribution.