2
$\begingroup$

I am comparing the correlation coefficient of variables during a period spanning 1963 - 2001 with the correlation coefficient of the same variables during a period spanning 2002 - 2005. Obviously, the first correlation coefficient is based on almost 10 times as many observations as the second. Can this explain the fact that the first correlation coefficient is higher than the second?

$\endgroup$
  • 2
    $\begingroup$ It's likely that values for 2002-2005 cover a narrower range than those for 1963-2001 and on both variables. The effect is to make noise more obvious than signal in the smaller set. It's this rather than the number of observations that is decisive. (It's not obvious that 39 years yield 40 times as many observations as 4 years: without further information we'd expect about 10 times more.) But plot the data to see. $\endgroup$ – Nick Cox Dec 2 '15 at 21:49
  • 1
    $\begingroup$ I second @NickCox's comment, particularly for practical data sets. Trends and autocorrelation in your variables can become important (if $x$ and $y$ are both increasing over time, but noisily, we expect a stronger correlation over longer time frames). Note that in theory, an increase in sample size is not necessarily linked to a stronger correlation coefficient. Sometimes the reverse is the case: in a sample size of two, the correlation is certain to be a perfect +1 or -1, unless your two $y$ observations happen to be the same (which for a continuous $y$ variable would generally be unusual). $\endgroup$ – Silverfish Dec 2 '15 at 21:54
  • 2
    $\begingroup$ However, if the true (population) correlation coefficient is non-zero, then increasing the sample size does make it more likely that you will detect a significant correlation, because even a weaker sample correlation would now count as significant (see e.g. this thread) $\endgroup$ – Silverfish Dec 2 '15 at 21:55
  • 1
    $\begingroup$ Revised wording: The effect is to make noise more obvious compared with signal in the smaller set, and thus to produce a lower correlation. Remember than a correlation is assessing how well a straight line would fit a set of points and increased scatter reduces the correlation. $\endgroup$ – Nick Cox Dec 2 '15 at 21:58
1
$\begingroup$

Its not the number of observations in itself that's matter, but a low number of observations do not represent the population very well. With observations from a longer timespan eventual trends will be clearer, and could produce spurious correlation.

You need to look at plots! Is there a visually obvious trend? Is the variance constant?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.