This question already has an answer here:

I am not able to understand logic behind coming up with this formula for covariance. We know that the sample covariance formula is:

$${\rm Cov}(x,y)=\frac{\sum(x_i - \bar{x})(y_i - \bar{y})}{n-1}$$

I am not able to understand the logic behind the numerator. Why are we multiplying the terms? I mean, I know we need to find the change in x is appearing along with change in y (if they co-vary), so this formula will give direction of that change in terms of sign of the number that you get after substituting the values in the formula. And also we will be able to make some comparisons, e.g., having a covariance of 50 is more evidence of x and y moving together than having it as 30. So their ordering will make sense. But if you just think of a single value of covariance without having to compare it with anything else, does it have any real intuitive meaning (just the number in itself)? For example, in case of variance or standard deviation one can easily see that it is average number of 'deviation from the mean' per data point in the sample. But here in case of covariance, I am not able to come up with any such intuitive understanding.


As I was adding tags for this question just now—I saw this as the tag description for the covariance the tag:

Covariance is a quantity used to measure the strength and direction of the linear relationship between two variables. The covariance is unscaled, & thus often difficult to interpret

So probably my difficult to understand covariance can be attributed to this.

marked as duplicate by whuber regression May 3 at 13:03

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

  • 2
    See here and here. Maybe this will help you answering your questions. – Stefan Mar 19 '17 at 3:44

I guess one justification maybe that for the same data points, if x and y move in directions with respect their means then the sign for the product will be negative. And since we are summing up the terms for all the data points -- if the negative elements will dominate, the entire expression will become negative and if the positive values dominate it becomes positive. So to say that the expression is positive is like saying -- we had more situations (data points) where x and y both moved in the same direction with respect to their own means and also if the expression is negative, we could say that we had more situations (data point) where x and y went to opposite direction with respect to their means. Dividing this entire thing by n-1 (or N for population) is still kind of wavy , I am still trying to make sense of the entire thing.

When we talk about variance for a single variable say x , we take the square of the distance because we are interested in the magnitude and not the direction or in simple words the spread of the distribution.

Covariance for a pair (x,y) is the magnitude and the sign by which it is distances itself from the mean . Since it involves the sign Covariance inherently gives us some information about the geographical distribution .

Magnitude

Now pertaining to your question the magnitude will tell us how that spread is located around the mean ( for eg. low covariance the pairs (x,y) are located near the mean high covariance means the pairs (x,y) are located farther away from the mean .

Not the answer you're looking for? Browse other questions tagged or ask your own question.