# When is it necessary to estimate the covariance matrix instead of calculating it directly?

The covariance of two random variables $(X_i,X_j)$ is given by $$Cov(X_i,X_j) = E[X_iX_j] - E[X_i]E[X_j],$$ where $E[X_i]$ is the expected value of $X_i$, or its mean.

The covariance matrix $\Sigma$, in turn, is defined as the covariances between all $\textbf{X} = (X_1,...,X_n)$ random variables, such that $\Sigma_{ij} = Cov(X_i,X_j)$.

In my studies, however, I've come in contact with techniques dedicated to estimate the covariance matrix (such as using maximum likelihood) but I couldn't find any real world problems requiring the use of such techniques. Moreover, as stated above, it is possible to calculate every element of $\Sigma$ using the covariance formula stated above.

My question is when is it necessary to estimate a covariance matrix instead of calculating it by directly using the covariance formula?

• Douglas: I changed the original title so it is less generic. Please free to amend it if you think it does not reflect your original theme but do be more specific that simply stating : Why do we need to estimate the covariance matrix? :) – usεr11852 May 4 '15 at 8:20
• I don't think the Mahalanobis distance has anything to do with my question, so I re-edited it in order to express myself more clearly. – Douglas De Rizzo Meneghetti May 4 '15 at 18:55
• If you happen to find an entire population of data in the wild that exactly follows a known joint distribution, feel free to calculate its exact covariance matrix. But you'll have your work cut out for you. Note that the maximum likelihood estimate just reduces to the sample covariance anyway. – shadowtalker May 4 '15 at 20:30

You always estimate the covariance matrix, there is no "formula" for it. For instance, the "covariance formula" which you refer to involves estimation of several quantities: $E[X_iX_j],E[X_j],E[X_i]$. Notice, that you don't know what are these values, so you actually estimate them. In this regard the "formula" is really an estimator. It's not necessarily the best estimator in all cases.