2
$\begingroup$

I had a discussion about covariance recently and it would be nice to hear your feedback about this.

Let's say we have a dataset of $n$ samples with $d$ attributes. For simplicity, let's say 3 of those $d$ attributes are e.g.,

$d_1$ = distance in miles
$d_2$ = temperature in Celsius
$d_3$ = elevation in meters

Now, let's assume the covariance between

$d_1$ and $d_2$ = $\sigma_{12} = 10$
$d_2$ and $d_3$ = $\sigma_{23} = 20$

The question is whether $d_2$ is more correlated with $d_3$ than $d_1$

So, my interpretation of covariance is that the covariance is a measure of how much 2 random variables vary together. A positive covariance indicates that both variables vary together in the same direction with respect to their sample mean.

In this case, we can't say that $d_2$ is more correlated with $d_3$ than $d_1$ since the attributes are measured on different "scales." All we can say is that there is a positive correlation in both cases.
However, if we would standardize all attributes, and we could make comparisons about the degree of correlation between attributes based on their magnitudes.

Does this make sense?

$\endgroup$
1
  • $\begingroup$ Standardizing variables converts covariance to correlation $\endgroup$
    – Glen_b
    Sep 22, 2014 at 3:41

1 Answer 1

3
$\begingroup$

(Note: Your notation for the covariance is weird. I'll use $\sigma_{12}$ for the covariance between $d_1$ and $d_2$; and $\sigma_1\sigma_2$ for the product of the two standard deviations.)

You're right, the covariance by itself only gives the direction of correlatedness. Without knowing the scales of your variables, you can't tell which variables are more correlated. But if you know the scales (measured by the standard deviation $\sigma_x$ of variable $d_x$), you can calculate the aptly named correlation coefficient $\rho_{12}$:

$$ \rho_{12} = \sigma_{12}/(\sigma_1 \cdot \sigma_2)$$

$\rho_{12}$ goes from -1 (totally reverse correlated) through 0 (not linearely correlated) to 1 (completely correlated). There are some other definitions for correlation coefficients, this here is Pearson's correlation coefficient.

$\endgroup$
1
  • $\begingroup$ Thanks, via the calculation you mention you'd basically obtain the/a "correlation matrix", right!? Also you are right about my weird notation of the covariance, I corrected it. $\endgroup$
    – user39663
    Sep 21, 2014 at 20:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.