Expected value of samples

Question

I would like to calculate a covariance matrix of my data sample. How do I calculate the expected value for it, e.g.:

$$ E[(X_{2}-\mu_{2})(X_{1}-\mu_{1})] $$

My problem is, that I don't understand how to calculate the expected value since on wikipedia's definition I need a probility function for it (where shall it come from?). The mean is no problem but it's not the mean what is required, isn't it?

But I know how to calculate the covariance matrix via R or Eigen, thus I guess it is doable somehow.. but how??

Two parameters of my data sample:

X1={2.75083, 2.75095, 2.75074, 2.75106, 2.75082, 2.75096}

X2={376.499, 374.8, 373.288, 382.299, 377.092, 374.752}

edit: I read a bit more carefully and when all probabilities are equal then it's just an average indeed. Is that correct?

Answer 1

Usually, given the data, you calculate the estimated mean. Let us call each sample from the true distribution (that has mean $\mu_1$) $x_i$. Let $\hat{\mu_1} = \frac{1}{n}\sum_{i=1}^{n}x_i$. In this equation, each term $x_i$ and $\hat{\mu_1}$ are random variables. By the linearity of expectation:

$ E[\hat{\mu_1}] = E[\frac{1}{n}\sum_{i=1}^{n}x_i] = \frac{1}{n}\sum_{i=1}^{n}E[x_i] = \frac{1}{n}nE[x_i] = E[x_i]$

But it is known that $x_i$ follows the true underlying distribution. By the law of large number, with enough samples your mean estimate $\hat{\mu_1} \rightarrow \mu_1$, as $n \rightarrow \infty$. It explains why $\frac{1}{n}\sum_{i=1}^{n}x_i$ is used to approximate $\mu_1 = E[x_i]$.

Thus, you have no probability function, but you have the samples. You approximate the functional values (expectation, mean, etc.) of the true probability distribution (where $x_i$'s come from) by using averaging on the data set you suspect comes from that true distribution. Thus, even though you do not know the true distribution, you can still compute what the mean should be on average.

For the covariance, variance, etc. you still use some form of averaging, but the arguments under sum-sign are different, and the divisor might be sth different from $\frac{1}{n}$, for example, it is $\frac{1}{n-1}$ for variance.