Comparing and Interpreting covariances

Question

I had a discussion about covariance recently and it would be nice to hear your feedback about this.

Let's say we have a dataset of $n$ samples with $d$ attributes. For simplicity, let's say 3 of those $d$ attributes are e.g.,

$d_1$ = distance in miles
$d_2$ = temperature in Celsius
$d_3$ = elevation in meters

Now, let's assume the covariance between

$d_1$ and $d_2$ = $\sigma_{12} = 10$
$d_2$ and $d_3$ = $\sigma_{23} = 20$

The question is whether $d_2$ is more correlated with $d_3$ than $d_1$

So, my interpretation of covariance is that the covariance is a measure of how much 2 random variables vary together. A positive covariance indicates that both variables vary together in the same direction with respect to their sample mean.

In this case, we can't say that $d_2$ is more correlated with $d_3$ than $d_1$ since the attributes are measured on different "scales." All we can say is that there is a positive correlation in both cases.
However, if we would standardize all attributes, and we could make comparisons about the degree of correlation between attributes based on their magnitudes.

Does this make sense?

Answer 1

(Note: Your notation for the covariance is weird. I'll use $\sigma_{12}$ for the covariance between $d_1$ and $d_2$; and $\sigma_1\sigma_2$ for the product of the two standard deviations.)

You're right, the covariance by itself only gives the direction of correlatedness. Without knowing the scales of your variables, you can't tell which variables are more correlated. But if you know the scales (measured by the standard deviation $\sigma_x$ of variable $d_x$), you can calculate the aptly named correlation coefficient $\rho_{12}$:

$$ \rho_{12} = \sigma_{12}/(\sigma_1 \cdot \sigma_2)$$

$\rho_{12}$ goes from -1 (totally reverse correlated) through 0 (not linearely correlated) to 1 (completely correlated). There are some other definitions for correlation coefficients, this here is Pearson's correlation coefficient.