9
$\begingroup$

It's well known that adding more regressors can only improve the $R^2$. What about the number of observations? Say you have a sample of size $N$, and you draw a random subsample of size $n < N$. How should, in principle, the $R^2$ change?

Two things are fairly intuitive:

  1. The closer the subsample size to the full sample, the lower the variance and the closer the average to that of the full sample. Naturally, once the sample is the same, the distribution of the average $R^2$ degenerates to that of the full sample.

  2. The smaller the subsample, the closer $R^2$ is to 1. In effect, when the number of observations is equal to that of variables, $R^2=1$.

Thus, the answer to the question seems to be along the lines of the following relationship

$$ R^2_\text{sub-sample} = (R^2_\text{full-sample})^{\dfrac{n-k}{N-k}} $$

where $k$ is the number of regressors. The curvature of the relationship might depend on multiple factors.

I had a go in R, and the above is sort of confirmed. The plots show the density plots of $R^2$ for $10,000$ sub-samples of a given dataset ($N$=790), for three different sub-sample size -- $n$ equal to $25,$ $50,$ and $500$ respectively. You can see that the $R^2$ is further away from the full-sample one (blue line) the smaller $n$ is.

So, how does the $R^2$ depend on sample size? Are you aware of a theorem about this?

$n = 25$

enter image description here

$n = 50$

enter image description here

$n = 500$

enter image description here

# Import data

library(data.table)
mydata <- fread('http://www.stats.ox.ac.uk/pub/datasets/csb/ch1a.dat')

# Compute benchmark R2 - all sample

fit <- lm(V3 ~ V2 + V4 + V5, data=mydata)
R2_benchmark <- summary(fit)$r.squared

# Compute R2 for M subsamples of size n

set.seed(263293) # obtained from www.random.org
M <- 10000
R2 <- numeric(M)
n <- 500

for(i in 1:M) {

  mysample <- mydata[sample(1:nrow(mydata), n, replace=FALSE),]
  fit <- lm(V3 ~ V2 + V4 + V5, data=mysample)
  R2[i] <- summary(fit)$r.squared

}

# Compare

plot(density(R2))
abline(v = R2_benchmark, col="blue")

t.test(R2,mu = R2_benchmark, alternative="two.sided")
$\endgroup$
4
  • 6
    $\begingroup$ No, the expectation of estimated $R^2$ will not change, but the variance of its estimate will decrease along the sample size. $\endgroup$
    – user158565
    Commented Dec 12, 2018 at 17:22
  • 5
    $\begingroup$ We need to take the statement "The smaller the subsample, the closer R2 is to 1" advisedly. Although it's true that the chance of a sample $R^2$ being close to $1$ might increase with smaller sample size, that's only because the sample $R^2$ becomes more variable as the sample size decreases. It definitely does not tend to grow closer to $1$! The theorems therefore focus on the distribution of a sample $R^2$ and, especially, on its variance. That distribution is directly related to the F ratio distribution of the regression F statistic. See your favorite regression text for details. $\endgroup$
    – whuber
    Commented Dec 12, 2018 at 17:37
  • $\begingroup$ I think that $R^2$ depends to the sample size. The expected value of $R^2$ when all regressors are independent and independent from the predicted value is: $\frac{k}{n-1}$. davegiles.blogspot.com/2013/10/… $\endgroup$
    – Simone
    Commented May 16, 2022 at 9:43
  • $\begingroup$ I think that your two fairly intuitive things are both wrong because if you keep the number of predictors constant the explained variance for $ n\to \infty$ just tend to the theoretical variance $\sigma^2$ of the error $\epsilon$ of the model, not to 0. $\endgroup$
    – No-one
    Commented May 25 at 0:01

2 Answers 2

10
$\begingroup$

No, the expectation of estimated $𝑅^2$ will not change, but the variance of its estimate will decrease along the sample size.

– user158565

We need to take the statement "The smaller the subsample, the closer $𝑅^2$ is to 1" advisedly. Although it's true that the chance of a sample $𝑅^2$ being close to 1 might increase with smaller sample size, that's only because the sample $𝑅^2$ becomes more variable as the sample size decreases. It definitely does not tend to grow closer to 1! The theorems therefore focus on the distribution of a sample $𝑅^2$ and, especially, on its variance. That distribution is directly related to the F ratio distribution of the regression F statistic. See your favorite regression text for details.

– whuber

$\endgroup$
1
  • 5
    $\begingroup$ I've copied these comments as a community wiki answer because they are, more or less, answers to this question. We have a dramatic gap between answers and questions. At least part of the problem is that some questions are answered in comments: if comments which answered the question were answers instead, we would have fewer unanswered questions. $\endgroup$
    – mkt
    Commented Jun 24, 2019 at 19:24
2
$\begingroup$

It seems that you are trying to describe what is known as the "Adjusted R-squared", which indeed depends on the number of observations n and the number of model parameters p:

$$R^2 = 1- \dfrac{SSRes}{SSTotal}$$ $$R^2_{adjusted} = 1- \dfrac{n-1}{n-p}\dfrac{SSRes}{SSTotal}$$

$\endgroup$

Your Answer

By clicking β€œPost Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.