Degrees of Freedom covariance I have troubles understanding the concept of degrees of freedom, especially in the case of the covariance. In the case of the standard deviation I think I understand why I have to devide the quatratic sum of the differences to the mean by (n-1). There is one degree less since I have to estimate the expected value. In the case of the covariance the denominator is still (n-1) but I have to estimate two expected values. Why isn't it n-2?
 A: The "degrees of freedom" argument is esoteric and only useful as a vague argument in the first chapter of a statistic text book because the actual reason cannot yet be explained to beginners. Unfortunately, this "explanation" leads to more confusion than plainly admitting that the beginner cannot yet understand it.
The actual reason for the correction is, that it makes the estimator unbiased. It is
$$E\left(\frac{1}{n}\sum_{i=1}^n (x_i-\overline{x})^2\right) = \frac{n-1}{n}\sigma^2$$
Multiply this equation with $n/(n-1)$, and you obtain an unbiased estimator for the variance.
Edit (for pundits): and here is the counterexample mentioned in the comments below that confutes the "degree of freedoms" argumentation for the denominator in the variance.
From the measured values $x_1,\ldots,x_n$, let us use only the first value $x_1$ as an estimator for the expectation value $\mu$. This estimator is unbiased because $E(x_1)=\mu$, although its variance is by a factor $n$ greater than the variance of $\overline{x}$, so this is merely a toy example. Nevertheless, we can build a variance estimator upon this estimator for $\mu$, namely
$$s'^2=\frac{1}{n}\sum_{i=1}^n (x_i-x_1)^2$$
Now let us check, whether this estimator is unbiased:
\begin{eqnarray*}
E(s'^2) & = & \frac{1}{n}\sum_{i=1}^n E\left((x_i-x_1)^2\right) = \frac{1}{n}\sum_{i=1}^n E\left([(x_i-\mu) - (x_1-\mu)]^2\right) \\
& = & \frac{1}{n}\sum_{i=1}^n \Big[ \underbrace{E\left((x_i-\mu)^2\right)}_{\sigma^2} - 2 E\left((x_i-\mu)(x_1-\mu\right) + \underbrace{E\left((x_1-\mu)^2\right)}_{\sigma^2}]
\end{eqnarray*}
As all $x_i$ are independent, it is
$$E\left((x_i-\mu)(x_1-\mu\right) = \left\{ \begin{array}{ll}
E\left((x_1-\mu)^2\right) = \sigma^2 & \mbox{ for }i=1 \\
E(x_i-\mu)\cdot E(x_1-\mu)=0 & \mbox{ for }i\neq 1
\end{array}\right.$$
It is thus
$$E(s'^2) = \sigma^2 -\frac{2}{n}\sigma^2 + \sigma^2 = 2\sigma^2\frac{n-1}{n}$$
An unbiased estimator for $\sigma^2$ is thus
$$\frac{n}{2(n-1)} s'^2 = \frac{1}{2(n-1)}\sum_{i=1}^n (x_i-x_1)^2$$
This in contradiction the the "degrees of freedom" argument, because only one variable ($x_1$) is reused in the formula and according to the "degress of freedom" heuristic, the denominator should be $(n-1)$ instead of $2(n-1)$.
Caveat: I have only proven that this is an unbiased estimator for $\sigma^2$, but have not yet verified this result with a Monte Carlo simulation.
A: Recall the error propagation formula for the addition and subtraction
operations:
$s[x+y]^2 = s[x]^2 + s[y]^2 + 2 s[x,y]$
and
$s[x-y]^2 = s[x]^2 + s[y]^2 - 2 s[x,y]$
Therefore:
$s[x+y]^2 - s[x-y]^2 = 4 s[x,y]$
Expanding the left hand side of this equation:
\begin{align*}
  s[x+y]^2 - s[x-y]^2 & = \frac{1}{n-1} \sum_{i} ([x_i+y_i]-[\bar{x}+\bar{y}])^2 - ([x_i-y_i]-[\bar{x}-\bar{y}])^2\\
  & = ... = \frac{1}{n-1} \sum_{i} \left( 4 x_iy_i - 4x_i\bar{y} - 4y_i\bar{x} + 4\bar{x}\bar{y}\right) \\
  & = \frac{4}{n-1} \sum_{i}(x_i-\bar{x})(y_i-\bar{y})
\end{align*}
which proves that
$s[x,y] = \frac{1}{n-1} \sum_{i}(x_i-\bar{x})(y_i-\bar{y})$
