5
$\begingroup$

There are a number of topics related to covariance on this site. What I am having trouble grasping: why is covariance a useful thing to calculate?

As far as I see it, covariance is not a helpful statistic. It is hard to interpret and it is not at all standardized (like correlation). It can be calculated on two variables with totally different measurement systems.

Does anyone have an example that could help elucidate the necessity of calculating covariance? Is it simply a means to an end in calculating parameters for regression?

$\endgroup$
  • $\begingroup$ Since covariances are variances of linear combinations, this question can also be understood as asking either (a) why linear combinations of variables are useful or (b) why variances are useful. Which of these aspects are you concerned about? $\endgroup$ – whuber Dec 11 '15 at 19:10
  • 1
    $\begingroup$ I would say part a) why linear combinations of variables are useful $\endgroup$ – ST21 Dec 11 '15 at 19:44
  • $\begingroup$ I strongly suspect if you randomly sampled mathematical formulas from books on statistics or probability, far more formulas would utilize covariance compared to correlation. $\endgroup$ – Matthew Gunn Dec 12 '15 at 9:36
6
$\begingroup$

Covariance matrix contains more information than correlation matrix:

  • You can derive a correlation matrix from a covariance matrix.
  • But you cannot derive a covariance matrix using only a correlation matrix! (You also would need the standard deviations.)

Covariance matrices contain all the information of: (i) a correlation matrix plus (ii) a standard deviation vector. In some sense, covariance matrices are the more compact, mathematically convenient object to work with.

Another example using covariance:

I'll bring up a simple finance example that doesn't obviously involve regression:

  • Let there be $n$ possible investment assets.
  • Let $\Sigma$ be the covariance matrix for the $n$ assets.
  • Let $w$ be a vector denoting portfolio weights on the $n$ assets.

Then portfolio variance is given by the matrix equation: $$ w^\top \Sigma w $$

You can't write this formula this succinctly using a correlation matrix.

A portfolio that minimizes variance would be a solution to: $$ \begin{align*} \text{minimize (over $w$) } \quad w^\top \Sigma w \\ \text{ subject to: }\quad\quad \quad w^\top 1 = 1 \end{align*}$$

Note this would be the same as minimizing the standard deviation of portfolio returns.

Covariance turns out to be a rather ubiquitous concept for any problem involving two or more random variables. It comes up all over the place. Better start getting used to it!

$\endgroup$
4
$\begingroup$

It all depends on how you write the parameters in question when describing a linear regression.

For random variables $X$ and $Y$, the best (in the sense of linear minimum-mean-square-error) estimate of $Y$ in terms of $X$ is commonly written as $$\hat{Y} = \rho\frac{\sigma_Y}{\sigma_X} (X-\mu_X) + \mu_Y\tag{1}$$ and then it is claimed that $\rho^2$ is the fraction of $\sigma_Y^2$ that has been "explained" by $X$. All this causes you to get all riled up and declare that the covariance is a totally useless concept. But some folks (not many) like to write $(1)$ as

$$\left(\hat{Y} -\mu_Y\right) = \left.\left. \frac{\operatorname{cov}(Y,X)}{\operatorname{var}(X)} \right(X-\mu_X\right)\tag{2}$$

and say that the deviation of the estimate $\hat{Y}$ from its mean and the deviation of $X$ from its mean have the same ratio as the covariance of $X$ and $Y$ and the variance of $X$, while the explained variance is just $\displaystyle \frac{\operatorname{cov}^2(Y,X)}{\operatorname{var}(X)}$. Would you be willing to listen to an argument from them that it is the covariance that is the more fundamental concept and that the correlation coefficient is just some gobbledygook of little interest? Why, it can't even make up its mind if its first name is Pearson or Spearman!

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.