1
$\begingroup$

From the formula of sample covariance, we see

$$\operatorname{Cov}(X,Y) = \frac{1}{m} \sum_{i=1}^m (x_i - \bar{x})(y_i - \bar{y})$$

where $\bar{x}, \bar{y}$ are the sample means. It seems like the sample size $m$ will have an impact on the final outcome. Is the intuition true?

$\endgroup$
  • $\begingroup$ The more data you have, the better information about their covariance or correlation. Hence what you are seeing. $\endgroup$ – Bach Feb 11 at 16:49
  • $\begingroup$ It depends on how you take the sample. Assuming it's a simple random sample, then it's the random selection that affects the outcome. Sample size affects it a little because this formula is a biased estimator of the population covariance but is asymptotically unbiased. That means the estimates with smaller samples will be a little low on the average, but the amount of that bias decreases with sample size. But is this what you're trying to get at, or did you mean to write "$1/(m-1)$" for the fraction? $\endgroup$ – whuber Feb 11 at 20:28
  • $\begingroup$ @whuber, yes this is exactly what I was getting at. I can now see that if it is a random sample, sample size will not impact covariance in a unidirectional way, unless it is a sample size of 1. $\endgroup$ – lvdp Feb 11 at 20:54
  • 1
    $\begingroup$ @Bach : The question appears to assume $(X,Y)$ is uniformly distributed on a finite set. Thus we have a census of the whole population rather than a sample. $\endgroup$ – Michael Hardy Feb 11 at 21:48
  • $\begingroup$ Would it be correct to call this formula 'population covariance' instead of 'sample covariance', for clarity. $\endgroup$ – Sergei Rodionov Feb 16 at 16:15
2
$\begingroup$

Remember that if you increase the number of data points then you will need actual new data points, so you will have more $x_i$ and $y_i$ values. This means that the value $m$ in your formula will change, but you will also now have additional terms in your sum, so that will change too. So yes, the sample covariance will change as you get more values (except in highly unusual cases) because you are using a different data set.

Assuming that the sequences $(X_1,Y_1), (X_2,Y_2), (X_3,Y_3), ...$ are exchangeable, as $m \rightarrow \infty$ the sample covariance will converge to the true covariance between the pairs of values. So in this case you will generally expect that as $m$ becomes larger the sample covariance will start to become more stable (i.e., change less) and will converge towards a fixed value.


Additional note: It is usual for the sample covariance to be defined with incorporation of Bessel's correction (i.e., using $m-1$ in the denominator instead of $m$). This is done to ensure that the sample covariance is an unbiased estimator of the true covariance.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.