0
$\begingroup$

Adjusted R-squared is defined as

r2_adj = 1-[((1-r2)(n-1))/(n-k-1)]

I have read at many places that adjusted R-squared is strictly less than R-squared, because k>0 and R-squared <=1. But I was just thinking upon this formula and it seems that adjusted R-squared will be greater than R-squared if somehow my number of data points n is less than number of independent variables k.

Is this an basic assumption too that n should always be greater than k+1?

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ When you have fewer data points than variables, what is the value of the unadjusted $R^2$?? $\endgroup$
    – whuber ♦
    Commented Mar 13, 2021 at 20:34
  • $\begingroup$ Its not dependent on variables or data points right? Lets assume R2 is 0, i.e. model is always predicting mean for all the instances. $\endgroup$
    – Utkarsh Prakash
    Commented Mar 14, 2021 at 4:05

1 Answer 1

Reset to default
2
$\begingroup$

Note first that there are many different R-squared estimators. For an overview, see the references below. Disclaimer: I am the author of the third one.

However, all of them indeed require that $p>N$, as otherwise, it is impossible to fit the regression model, in the sense that no unique regression coefficients exist. If you for example try to fit a model with $11$ predictors and $10$ samples, using the lm function in R, it will not provides estimates for two predictors and also no standard errors for any coefficient.

While you technically can get an R-squared from such a model, it is guaranteed to be $1$ and thus meaningless, as will any adjustment of this value.See also the comment by whuber.

You can of course obtain a prediction function for the $p>N$ setting using a different technique, for example, ridge regression. However, all the adjusted R-squared formulas only work for $R^2$ estimates as obtained by ordinary least squared regression. You could fall back to predictive $R^2$ for this case but this estimate something else than adjusted-$R^2$, see my answer to another question: https://stats.stackexchange.com/a/518400/30495

References

$[1]$: Shieh G (2008): Improved shrinkage estimation of squared multiple correlation coefficient and squared cross-validity coefficient. Organizational Research Methods, 11(2): 387-407 (link)

$[2]$: Yin P, Fan X (2001): Estimating $R^2$ shrinkage in multiple regression: A comparison of different analytical methods. The Journal of Experimental Education, 69(2): 203-224 (link)

$[3]$: Karch J (2020): Improving on adjusted R-squared. Collabra: Psychology (2020) 6 (1): 45. (link)

Improve this answer
$\endgroup$
2
  • $\begingroup$ The problem is the opposite of your characterization: when the rank of the model matrix equals $n,$ there will be a perfect fit (and probably many of them), making $R^2=1,$ whence any formula of the form $1 - (1-R^2)*\text{positive constant}$ will give $1$ for its adjusted version. The attempt to divide by $n-k-1$ in the formula is a strong hint that using $R^2$ and attempting to adjust it in such circumstances makes no sense. $\endgroup$
    – whuber ♦
    Commented Apr 6, 2021 at 15:00
  • 1
    $\begingroup$ This just seems to be a misunderstanding in what I meant with it is impossible to fit the regression model. I edited for clarification. $\endgroup$
    – Julian Karch
    Commented Apr 6, 2021 at 15:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.