3
$\begingroup$

Suppose we have a model with $n$ data points and $p$ predictors. The model has an $R^2_{\text{adjusted}}=q$.

What sort of distribution does the change in $R^2_{\text{adjusted}}$ have when a random Gaussian predictor is added to the model?

I am assuming that the expected value is zero, since $R^2_{\text{adjusted}}$ is supposed to adjust for predictors that have no explanatory power. But I can't find the answer anywhere, and I'm having trouble working it out myself. ...

$\endgroup$
  • $\begingroup$ Wikipedia confirms your supposition that the expectation will be zero. The distribution, as a linear transformation of a ratio of sums of squares, tends to be strongly skewed. You might consider beginning with the case of no predictors (except a constant) and consider what happens when you include one independent predictor. $\endgroup$ – whuber Jun 26 '13 at 4:15
1
$\begingroup$

Here's a quick simulation in R that gives you a feel for the distribution of $R^2_{adj}$ when adding a single random normal variable.

set.seed(1234)
adj_r_simulation <- function(n=100) {
    y <- rnorm(n)
    x <- rnorm(n)
    fit <- lm(y~x)
    summary(fit)$adj.r.squared
}

# Simulate adjusted r-square values
simulations <- 1000
n <- 100
x <- sapply(seq(simulations), function(X) adj_r_simulation(n=n))

# Display results
mean(x)
sd(x)
plot(density(x), 
     main=paste0("n=",n, "; simulations=", simulations),
     xlab='Adjusted r-squared')
abline(v=mean(x))

The simulation above simulates two independent normal variables, x and y of sample size n=100. It then fits a linear model predicting y from x and returns $R^2_{adj}$. This process is repeated 1000 times and the values are saved. The sample mean of simulated $R^2_{adj}$ values is close to zero:

> mean(x)
[1] 0.0002181863
> sd(x)
[1] 0.01425647

Below you can see the density plot of the simulated values of $R^2_{adj}$. This illustrates whuber's point about the distribution being highly skewed.

distribution of adjusted r squared when adding independent normal variable

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.