I want to break down this statement:

$|E[(X - \bar{x})(Y - \bar{y})]|^2 = |<X - \bar{x}, Y - \bar{y}>|^2$

I am not familiar with expectation values being broken down into vectors. I only know that by definition $\displaystyle E[(X - \bar{x})^2] = \sum_{i=1}^{n} \frac{(x_{i} - \bar{x})^2}{n}$ and I would like to know how expectation values can be viewed as vectors specifically in the context of inner products like $E[(X - \bar{x})^2] = <X-\bar{x}, X-\bar{x}>$ Also whatever happened to the n?

My other question is how do I view covariance as a vector? I know that covariance is $E[XY] - E[X]E[Y]$ so how do I rewrite that in vector form?


1 Answer 1


The "by definition" equality you write does not hold.

$$\displaystyle E[(X - \bar{x})^2] = \int_{S_X}(x - \bar{x})^2f_X(x) dx $$ is the correct definition for continuous r.v.'s , with $S_X$ the support of $X$ and $f_X(x)$ the pdf of $X$. For discrete random variables

$$E[(X - \bar{x})^2] = \sum_{S_X}(x - \bar{x})^2p_X(x) $$ Now IF the $x$'s can be viewed as realization of the same ergodic and stationary stochastic process, THEN $\frac {1}{n}\sum_{i=1}^{n} (x_{i} - \bar{x})^2$ is a consistent estimator of $E[(X - \bar{x})^2]$.

The expected value operator is applied to each element of any vector-matrix. If

$$A=\left[\begin{matrix} a_{11} &...& a_{1n}\\ ... & ...& ... \\ a_{k1} &...&a_{kn} \end{matrix}\right]$$

then $$E(A) = \left[\begin{matrix} E(a_{11}) &...& E(a_{1n})\\ ... & ...& ... \\ E(a_{k1}) &...&E(a_{kn}) \end{matrix}\right]$$

If $\mathbf x$ and $\mathbf y$ are two $n\times 1$ column vectors, then (prime denoting the transpose) $$ \operatorname{Cov}(\mathbf x,\mathbf y) = E(\mathbf x \mathbf y') - E(\mathbf x)\Big[E(\mathbf y)\Big]'$$

This is the expression for the covariance of two random vectors. If you want the covariance matrix of two samples, look up this answer in math.SE

  • $\begingroup$ Your answer is informative and introduced a lot of new ideas, but I'm still wondering whatever happened to the pdf or "n" in the case of discrete r.v.s when dealing with expected values expressed as vectors? I have seen $E[XY]$ expressed as $ <X,Y>$ $\endgroup$
    – Person
    Oct 21, 2013 at 2:54
  • $\begingroup$ The size of the sample $n$ is used when we are estimating the expected value from a sample. When we define the expected value the "weight" factor is the pmf itself. Indeed, the expected value is a "weighted" average, while the sample mean is an "unweighted" average (nevertheless, the latter is a consistent estimator of the former). As for the second issue, you are confused because in other scientific fields, the symbol $< >$ is used instead of the symbol $E$ -it means exactly the same thing. $\endgroup$ Oct 21, 2013 at 8:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.