# Confusion with covariance

For distributions of random variables X and Y, their covariance can be defined as the difference between the multiplication of X and Y, normalized by their joint probability and the multiplication of X and Y normalized by their marginal probabilities:

$$\operatorname{Cov}(X,Y) = E(XY) - E(X)E(Y) = \sum_{x,y}p(x,y)xy - \left(\sum_xp(x)x\right)\cdot \left(\sum_yp(y)y\right)$$

From sample data for X and Y, covariance can be defined as how on average X and Y differ from their respective means:

$$\operatorname{COV}(X,Y)= \frac{1}{N}\sum_i^N (x_i-\bar{x}) (y_i-\bar{y})$$

According to the second definition, if higher values of Y occur for higher values of X, then the resultant covariance is high.

How this translates to the first definition?

The first definition says that X and Y will have high covariance when joint probabilities for pairs of values will have higher values than their individual probabilities, regardless of the actual values.

I tried with the following values:

x = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9], y = [0, 1, 2, 3, 4, -1, -2, -3, -4, -5]

And got a surprisingly large covariate value of -6.9 (correlation -.7). I do not understand why?

• The first one is the parameter or the function of parameters. The second one is the estimate of the first one. – user158565 Nov 6 '18 at 22:41

The two versions of covariance are more alike than it appears at first glance. First, the covariance between random variables $$X$$ and $$Y$$ $$\operatorname{Cov}(X,Y)=E(XY) - E(X)E(Y)$$ can also be written, after a bit of algebra, $$\operatorname{Cov}(X,Y)=E\left[\left(X-E(X)\right)\left(Y-E(Y)\right)\right].$$ Second, given a list of $$n$$ pairs $$(x_1,y_1),(x_2,y_2),\ldots(x_n,y_n)$$, it's straightforward to check that the sample covariance of this list is exactly the covariance between random variables $$X$$ and $$Y$$, where the random variables are defined to place mass $$\frac1n$$ at each of the pairs $$(x_1,y_1),(x_2,y_2),\ldots(x_n,y_n)$$.