# multivariate Gaussian/normal distribution- sigma covariance and eigenvector

For a Multivariate Gaussian Distribution

I am a bit confused about the derivation of sigma

I am confused that I still cannot get the details of the derivation of sigma.

Could anyone help me with the process from having known $$\Sigma u_i=\lambda_i u_i$$ and $$u_i$$is orthonormal, to the result of $$\Sigma=\sum^D_i \lambda_i u_i u_i^T$$?

From $$\Sigma u_i = \lambda_i u_i$$, we can get by multiply $$u_i^T$$ on the right sides $$\Sigma u_i u_i^T = \lambda_i u_iu_i^T, \quad i=1, \ldots, D.$$ Now, let us sum up $$D$$ equations above, which leads to $$\Sigma \sum_i u_i u_i^T = \sum_i \lambda_i u_iu_i^T.$$ Let $$U=[u_1, \ldots, u_D]$$, a column-binded matrix from $$\{u_i\}_{i=1}^D$$. If $$u_i$$'s are orthonormal (i.e. $$\sum_i u_i u_i^T=U U^T = I$$), we now conclude $$\sum_i \lambda_i u_iu_i^T = \Sigma \sum_i u_i u_i^T=\Sigma.$$

As whuber suggested, the eigenvectors should satisfy orthonormality, not just orthogonality. OP would be better to check this condition.

Hope this helps.

• +1 -- but your argument is a little incomplete. Orthonormality of the $u_i$ does not (in itself) imply $UU^\prime = I:$ you also need that there are $D$ of the $u_i.$ I realize this is implicit in your argument, but making it explicit may assist the reader in understanding the key ideas.
– whuber
Jan 31 '20 at 23:23
• @whuber Agreed. I made an edit based on your suggestion. Thanks! Feb 1 '20 at 3:23
• @inmybrain Thank you for your prompt and useful answer. I think ui(i=1,...,D) is the elements of the vector U. Thus, the transpose of ui is equal to ui,because ui itself is a scalor not a vector. Could you please tell me whether my thoughts is correct? U is the mean so it is a D*1 vector, and vector x as observation, is also a D*1 vector, the sigma covariance matrix is a D*D matrix, is that right？ Feb 1 '20 at 15:05
• @CiciYang I'm afraid to say it is not. $u_i$ is a $D$-dimensional vector, as $x$ is. $\Sigma$ is a $D \times D$ matrix. Feb 1 '20 at 23:20
• @@inmybrain Can you please explain why (sum uiui(transpose)=UU(transpose)? While using matrix multiplication I can get UU(transpose)=I, I still can not get why sum uiui(transpose)=UU(transpose) exists. I assume that U is a D*D matrix with D column and observations. Is that right? Is there an intuitive way to explain why the sum of the uiui(T)=UU(T)? Thank you. Feb 2 '20 at 6:14