For distributions of random variables X and Y, their covariance can be defined as the difference between the multiplication of X and Y, normalized by their joint probability and the multiplication of X and Y normalized by their marginal probabilities:

$$\operatorname{Cov}(X,Y) = E(XY) - E(X)E(Y) = \sum_{x,y}p(x,y)xy - \left(\sum_xp(x)x\right)\cdot \left(\sum_yp(y)y\right)$$

From sample data for X and Y, covariance can be defined as how on average X and Y differ from their respective means:

$$\operatorname{COV}(X,Y)= \frac{1}{N}\sum_i^N (x_i-\bar{x}) (y_i-\bar{y}) $$

According to the second definition, if higher values of Y occur for higher values of X, then the resultant covariance is high.

How this translates to the first definition?

The first definition says that X and Y will have high covariance when joint probabilities for pairs of values will have higher values than their individual probabilities, regardless of the actual values.

I tried with the following values:

x = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9], y = [0, 1, 2, 3, 4, -1, -2, -3, -4, -5]

And got a surprisingly large covariate value of -6.9 (correlation -.7). I do not understand why?

  • $\begingroup$ The first one is the parameter or the function of parameters. The second one is the estimate of the first one. $\endgroup$
    – user158565
    Nov 6, 2018 at 22:41

1 Answer 1


The two versions of covariance are more alike than it appears at first glance. First, the covariance between random variables $X$ and $Y$ $$ \operatorname{Cov}(X,Y)=E(XY) - E(X)E(Y) $$ can also be written, after a bit of algebra, $$ \operatorname{Cov}(X,Y)=E\left[\left(X-E(X)\right)\left(Y-E(Y)\right)\right]. $$ Second, given a list of $n$ pairs $(x_1,y_1),(x_2,y_2),\ldots(x_n,y_n)$, it's straightforward to check that the sample covariance of this list is exactly the covariance between random variables $X$ and $Y$, where the random variables are defined to place mass $\frac1n$ at each of the pairs $(x_1,y_1),(x_2,y_2),\ldots(x_n,y_n)$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.