Expectation values as vectors? I want to break down this statement:
$|E[(X - \bar{x})(Y - \bar{y})]|^2 = |<X - \bar{x}, Y - \bar{y}>|^2$ 
I am not familiar with expectation values being broken down into vectors. I only know that by definition $\displaystyle E[(X - \bar{x})^2] = \sum_{i=1}^{n} \frac{(x_{i} - \bar{x})^2}{n}$ and I would like to know how expectation values can be viewed as vectors specifically in the context of inner products like $E[(X - \bar{x})^2] = <X-\bar{x}, X-\bar{x}>$  Also whatever happened to the n?
My other question is how do I view covariance as a vector? I know that covariance is $E[XY] - E[X]E[Y]$ so how do I rewrite that in vector form? 
 A: The "by definition" equality you write does not hold.
$$\displaystyle E[(X - \bar{x})^2] = \int_{S_X}(x - \bar{x})^2f_X(x) dx $$
is the correct definition for continuous r.v.'s , with $S_X$ the support of $X$ and $f_X(x)$ the pdf of $X$.
For discrete random variables
$$E[(X - \bar{x})^2] = \sum_{S_X}(x - \bar{x})^2p_X(x) $$
Now IF the $x$'s can be viewed as realization of the same ergodic and stationary stochastic process, THEN $\frac {1}{n}\sum_{i=1}^{n} (x_{i} - \bar{x})^2$ is a consistent estimator of $E[(X - \bar{x})^2]$.
The expected value operator is applied to each element of any vector-matrix. If
$$A=\left[\begin{matrix}
a_{11} &...& a_{1n}\\
... & ...& ... \\
a_{k1} &...&a_{kn}
\end{matrix}\right]$$
then 
$$E(A) = \left[\begin{matrix}
E(a_{11}) &...& E(a_{1n})\\
... & ...& ... \\
E(a_{k1}) &...&E(a_{kn})
\end{matrix}\right]$$
If $\mathbf x$ and $\mathbf y$ are two $n\times 1$ column vectors, then (prime denoting the transpose)
$$ \operatorname{Cov}(\mathbf x,\mathbf y) = E(\mathbf x \mathbf y') - E(\mathbf x)\Big[E(\mathbf y)\Big]'$$
This is the expression for the covariance of two random vectors. If you want the covariance matrix of two samples, look up this answer in math.SE
