2
$\begingroup$

When calculating the sample covariance, why do we divide by $n-1$ instead of $n-2$? Don't we lose two degrees of freedom since we need to calculate two sample means? For example, when estimating the variance for a Bayes classifier, we divide by $n-K$ where $K$ is the number of classes since we use $K$ sample means in the calculation.

Could someone please explain this in terms of degrees of freedom?

UPDATE

The other answers on this site don't quite make sense to me. So for clarification, I would like to extend this question such that I believe it is sufficiently different to not be marked as a duplicate.

The definition of degrees of freedom as per my understanding is $df = n-p$, where $p$ is the number of parameters calculated en route to the current calculation. I guess the issue boils down to what a "parameter" is considered to be. Clearly in the case of covariance, the individual sample means are not each treated as a parameter, since $df=n-1$. Which leads me to hypthothesize the crude definition that $df=n-p$, where $p$ is the number of sequentially dependent parameters.

For example, in the case of covariance, both sample means can be calculated independently of each other. But in calculating residual standard error for simple regression, $df=n-2$ since the intercept term can only be calculated knowing the estimated gradient - or it is sequentially dependent on the estimated gradient.

Is this a valid definition? If so, does that mean that:

$f(X,Y)=\sum_{i=0}^n(X_{i}-\overline{X})^2+\sum_{i=0}^n(Y_{i}-\overline{Y})^2$ has $df=n-1$ also?

If not - with reference to both covariance and the above equation - why so?

$\endgroup$
  • $\begingroup$ In response to your edit I have provided a link to a thread containing extensive discussions of degrees of freedom and their relationship to parameter counts. $\endgroup$ – whuber Sep 25 at 17:51