Why is Covariance Useful?

Question

There are a number of topics related to covariance on this site. What I am having trouble grasping: why is covariance a useful thing to calculate?

As far as I see it, covariance is not a helpful statistic. It is hard to interpret and it is not at all standardized (like correlation). It can be calculated on two variables with totally different measurement systems.

Does anyone have an example that could help elucidate the necessity of calculating covariance? Is it simply a means to an end in calculating parameters for regression?

Answer 1

Covariance matrix contains more information than correlation matrix:

• You can derive a correlation matrix from a covariance matrix.
• But you cannot derive a covariance matrix using only a correlation matrix! (You also would need the standard deviations.)

Covariance matrices contain all the information of: (i) a correlation matrix plus (ii) a standard deviation vector. In some sense, covariance matrices are the more compact, mathematically convenient object to work with.

Another example using covariance:

I'll bring up a simple finance example that doesn't obviously involve regression:

• Let there be $n$ possible investment assets.
• Let $\Sigma$ be the covariance matrix for the $n$ assets.
• Let $w$ be a vector denoting portfolio weights on the $n$ assets.

Then portfolio variance is given by the matrix equation: $$ w^\top \Sigma w $$

You can't write this formula this succinctly using a correlation matrix.

A portfolio that minimizes variance would be a solution to: $$ \begin{align*} \text{minimize (over $w$) } \quad w^\top \Sigma w \\ \text{ subject to: }\quad\quad \quad w^\top 1 = 1 \end{align*}$$

Note this would be the same as minimizing the standard deviation of portfolio returns.

Covariance turns out to be a rather ubiquitous concept for any problem involving two or more random variables. It comes up all over the place. Better start getting used to it!

Answer 2

It all depends on how you write the parameters in question when describing a linear regression.

For random variables $X$ and $Y$, the best (in the sense of linear minimum-mean-square-error) estimate of $Y$ in terms of $X$ is commonly written as $$\hat{Y} = \rho\frac{\sigma_Y}{\sigma_X} (X-\mu_X) + \mu_Y\tag{1}$$ and then it is claimed that $\rho^2$ is the fraction of $\sigma_Y^2$ that has been "explained" by $X$. All this causes you to get all riled up and declare that the covariance is a totally useless concept. But some folks (not many) like to write $(1)$ as

$$\left(\hat{Y} -\mu_Y\right) = \left.\left. \frac{\operatorname{cov}(Y,X)}{\operatorname{var}(X)} \right(X-\mu_X\right)\tag{2}$$

and say that the deviation of the estimate $\hat{Y}$ from its mean and the deviation of $X$ from its mean have the same ratio as the covariance of $X$ and $Y$ and the variance of $X$, while the explained variance is just $\displaystyle \frac{\operatorname{cov}^2(Y,X)}{\operatorname{var}(X)}$. Would you be willing to listen to an argument from them that it is the covariance that is the more fundamental concept and that the correlation coefficient is just some gobbledygook of little interest? Why, it can't even make up its mind if its first name is Pearson or Spearman!

Answer 3

I had the same question. Seems that the value of the covariance, by itself, is meaningless. The only thing you can tell by the covariance itself is that, is the covariance is a positive number the populations show similar behavior and it is a negative number the populations show opposing behavior.

However, the value of the covariance is a necessary component of other calculations. In finance, the covariance (of a share and the market) / the variance in market = the Beta value. which is a useful thing.

So, by itself the magnitude of the covariance is meaningless but in combination with other things it provides meaning.