relation between R-squared and sample size

Question

Here is the question:

Suppose $X, Y$ are independent $N(0,1)$ random variables. And take the regression of $Y$ against $X.$ What is the relationship between $R^2$ and sample size approximately?

First I am not sure if "independent" is a typo in the original question. In ordinary least squares (OLS), there is no specific relation between $R^2$ and sample size.

Here R2 is defined as: $$R2 = 1 - \dfrac{RSS}{TSS}.$$ $$RSS = \sum(y_i-\hat{y})^2,\ TSS = \sum(y_i-\bar{y})^2.$$ And in the simple linear regress with interception, R2 is exactly correlation between X and Y.

Answer 1

As sample size increases, $R^2$ decreases toward zero.

Since the two variables are independent, they have zero correlation and, thus, zero squared correlation. With a small sample size, some bad luck might result in there being a high empirical correlation. As the sample size increase, however, the empirical correlation will tend toward the true value of zero, hence the squared correlation tending toward zero.

Let’s look at it in a simulation.

library(ggplot2)
set.seed(2023)
Ns <- rep(seq(3, 550, 1), 4) 
r2 <- rep(Ns)
for (i in 1:length(Ns)){
    x <- rnorm(Ns[i])
    y <- rnorm(Ns[i])
    r2[i] <- (cor(x, y))^2
}
d <- data.frame(
    sample_size = Ns,
    r2 = r2
)
ggplot(d, aes(x = sample_size, y = r2)) +
    geom_point()
    xlab(“Sample Size”) +
    ylab(“R^2”)