I'm struggling to understand this presentation of covariance. It says:
The variance-covariance matrix (or simply the covariance matrix ) of a random vector $\overline{X}$ is given by:
$$Cov(\overline{X})=E[\overline{X}\overline{X}^T]-E\overline{X}(E\overline{X})^T$$
First of all , what does he/she mean with covariance MATRIX of A VECTOR ? Variance of a vector (covariance with itself) ? But isn't that supposed to be a scalar, not a matrix ?!
Second, if you multiply $\overline{X}$ with $\overline{X}^T$, you get a matrix. How do you get the expected value ($E[\overline{X}\overline{X}^T]$) of that matrix?? It doesn't have multiple layers!!
Then, if you take the expected value ($E\overline{X})$ of $\overline{X}$, it's a scalar! What is the point in multiplying it with its transpose? The transpose of a scalar is itself!
I know that the usual definition of covariance is
$$cov(x.y)=\frac{\sum_{i=1}^{n}(x_i-\overline{x})(y_i-\overline{y})}{N-1}$$
THIS makes perfect sense, but the earlier presentation with expected values and vectors is confusing!