Is there anything interesting to be taken from the fact that $E[(X-E[X])(Y-E[X])] = E[(X-E[X])(Y-E[Y])]$?

Question

While playing around with the formula for covariance, I discovered something I wasn't expecting. Replacing the $E[Y]$ in the definition of covariance with an $E[X]$ appears to simplify back down to the covariance via the alternative formula for covariance.

$$\begin{align} E[(X-E[X])(Y-E[X])] &= E[XY -E[X]Y -E[X]X + E[X]E[X]]\\ &= E[XY] - E[E[X]Y]-E[E[X]X] + E[X]^2\\ &= E[XY] -E[X]E[Y] -E[X]^2 + E[X]^2 \\ &= E[XY] - E[X]E[Y] \\ &= COV[X,Y] \\ &= E[(X-E[X])(Y-E[Y])] \end{align}$$

Is this a well known manipulation, and is there a more meaningful way to explain it than pure algebraic derivation?

One would think measuring the "center" of $Y$ "incorrectly" would impact the covariance, but it changes nothing when the center is artificially moved to $E[X]$. What gives?

Answer 1

Wow this got me thinking for awhile. So I tried some different manipulations to see where things goes:

It seems you can get to the covariance in many ways but I notice the pattern that at least one of the variables must be centered properly while the other can be shifted by either it's center, the other's center, arbitrary constant, or none at all. $$ \begin{align} E[(X-E(X))(Y-E(X)]&=E[XY]-E[X]E[Y]\\ E[(X-E(Y))(Y-E(Y)]&=E[XY]-E[X]E[Y]\\ E[(X-E(X))Y]&=E[XY]-E[X]E[Y]\\ E[X(Y-E(Y))]&=E[XY]-E[X]E[Y]\end{align} $$ also just tried what happens if there isn't any proper centering at all and it doesn't work $$ E[(X-E(Y))(Y-E(X)]\neq E[XY]-E[X]E[Y] $$ Intuitively, covariance is the measure of the direction of the relationship of two variables. Hence the need for centering to help see the direction. If values go below the mean (going negative) is paired with smaller values then the overall sum would be positive direction because of the smaller negative values. Then the other way around if it's a negative direction. And it seems the only requirement is at least centering one of them.

I tried to see it numerically as well and it seems true

X=rnorm(20,10,2)
Y=2*X+rnorm(20)

#Y: other's center
sum=0
for(x in 1:20){
  sum=sum+(X[x]-mean(X))*(Y[x]-mean(X))
}
#sum=160.7643


#Y: no centering
sum=0
for(x in 1:20){
  sum=sum+(X[x]-mean(X))*(Y[x])
}
#sum=160.7643


#Y: arbitrary shifting
sum=0
for(x in 1:20){
  sum=sum+(X[x]-mean(X))*(Y[x]-10230)
}
#sum=160.7643

Answer 2

Like @JarleTufto pointed out, adding zero allows us to express the form $E[(X-c_X)(Y-c_Y)]$ as covariance plus some other terms. Add zero: $$E[(X-E[X]+E[X]-c_X)(Y-E[Y]+E[Y]-c_Y)]$$ $$E[((X-E[X])+(E[X]-c_X))((Y-E[Y])+(E[Y]-c_Y))]$$ Expand: $$E[(X-E[X])(Y-E[Y])+(E[X]-c_X)(Y-E[Y])+(E[Y]-c_Y)(X-E[X])+(E[X]-c_X)(E[Y]-c_Y)]$$ Addition Rule: $$E[(X-E[X])(Y-E[Y])]+E[(E[X]-c_X)(Y-E[Y])+(E[Y]-c_Y)(X-E[X])]+E[(E[X]-c_X)(E[Y]-c_Y)]$$ Of course $Y-E[Y]$ and $X-E[X]$ have expected values of $0$, so the middle term evaluates to zero, and our expression becomes: $$COV[X,Y]+0+(E[X]-c_X)(E[Y]-c_Y)$$ If $c_X=E[X]$ then the whole thing simplifies to basic covariance again.

Mis-measuring the centers of the distributions by $E[X]-c_X$ and $E[Y]-c_Y$ results in the covariance plus an error term (both errors multiplied together). If one of the errors is zero, then you get covariance again.