Is there any relationship between $R^2$ and sample size - does the $R^2$ increase with sample size? And does the adjusted $R^2$?
-
2$\begingroup$ Whether the R^2 changes on average with sample size can't be answered without some assumptions about the process you're sampling. Since the X's are fixed in regression, this presents some difficulties in guessing what sampling circumstances you might mean. However, it's possible to specify reasonable circumstances in which the $R^2$ should approach a 'population' value. $\endgroup$– Glen_bCommented Sep 10, 2013 at 3:57
1 Answer
It depends on whether you are interested in $r^2$, the sample correlation coefficient, or the $R^2$ multiple correlation coefficient, used to assess the performance of regressions.
Both $r^2$ and adjusted $r^2$ are negatively biased--that is, the sample values are slightly smaller than the corresponding population value--but the adjusted formula is somewhat less biased. In addition to the sample size, the amount of bias depends on the value, with $r^2$ near zero and one showing the least bias and those near 0.6-0.8 showing the most bias.
Table 1 of a paper by Zimmerman, Zumbo, and Williams (2003) illustrates the bias as a function of sample size and correlation value. Elsewhere in the paper, they show simulation data indicating that the Fisher and Olkin and Pratt adjusted $r^2$ reduce this bias considerably.
There is also a decent amount of work looking at "$R^2$ shrinkage", which is a related phenomena that comes up a lot in regression-related contexts, but has the opposite sign (it is positively-biased, and adjustments bring it back down). Yin and Fan (2001) have a fairly comprehensive comparison of methods for estimating it, and Page 3/205 has some citations to descriptions of the problem.
Finally, you should be aware that there are lots of methods for adjusting $r^2$/$R^2$ (in fact, there are even multiple ($\ge3$) versions of the Olkin and Pratt adjustment formula floating around, some of which correct for the number of parameters), so it might help to be more specific about whatever you have in mind
-
$\begingroup$ Adjusted $R^2$ < $R^2$. If $R^2$ is negatively biased then adjusted $R^2$ must be more biased, not less. Figure 1 in the Zimmerman et al paper you linked to shows the bias of rank correlation. $\endgroup$ Commented Sep 11, 2013 at 3:31
-
$\begingroup$ Gah, that should have been Table 1 instead of Figure 1 (which has both Pearson $r$ and the rank correction). Are you sure that adjusted $r^2$ is biased more, rather than less? I thought (one) of the reasons to adjust was to reduce that bias, and Figure 2 (really, not the table!) seems to confirm that. $\endgroup$ Commented Sep 11, 2013 at 18:35
-
$\begingroup$ Zimmerman et al are talking about $r$, which is biased low and needs to be increased, whereas the OP and Yin & Fan are talking about $R^2$, which is biased high and needs to be reduced. $\endgroup$ Commented Sep 12, 2013 at 5:04