3
$\begingroup$

First question: is it possible for a multiple regression model to have "big" and significant coefficients but a low $R^2$ value?

Let's say the value of $R^2$ is 0.0005 and my coefficient of interest is 0.6, ignoring the coefficients of all other variables I adjust for. And the dependent variable has

range = [0,5]

mean = 2.5

variance = 4

Second question: if that's possible, does the low $R^2$ value affect the validity and the interpretation of my coefficient of interest?

Edit: I might be using the term validity loosely here. But my understanding is $R^2$ quantifies how much variation in the dependent variable explained by the model. So, does it make sense that I interpret the effect of the coefficient as usual (e.g. holding other things constant, 1 unit increase in my variable of interest increases the value of dependent variable by 0.6 on average) even though it explains very little of variation in the dependent variable?

$\endgroup$
  • $\begingroup$ To your Ist question ; yes it is possible. 2nd question -. R-SQUARE DEPENDS ON VALIDITY. I t is not the vice-a-versa. $\endgroup$ – Subhash C. Davar Jul 19 at 4:14
  • 1
    $\begingroup$ I might be using the term validity loosely here. But to clear things up, because r-squared quantify explained variance of the dependent variable by the model, does it make sense that I interpret the effect of the coefficient as usual (e.g. holding other things constant, 1 unit increase in my variable of interest increases the value of dependent variable by 0.6 on average) even though it doesn't explain much of variation in the dependent variable? $\endgroup$ – ChasNew Jul 19 at 4:26
  • $\begingroup$ even though it doesn't explain much of variation in the dependent variable? It is obscure to me $\endgroup$ – Subhash C. Davar Jul 19 at 5:22
  • 1
    $\begingroup$ 1 unit increase in my variable of interest increases the value of dependent variable by 0.6 on average) It is incorrect interpretation. $\endgroup$ – Subhash C. Davar Jul 19 at 5:25
6
$\begingroup$
  1. Yes, it's completely possible to get large coefficients and small $R^2$ values. Consider that the coefficient values depend on units, while $R^2$ does not. So if a model with length as a predictor switches from using kilometres to millimetres, the coefficients will increase by a factor of $10^6$ without changing the $R^2$.
  2. Yes, your interpretation is correct. If your coefficient is 0.6, the interpretation would be that an increase in the predictor of 1 unit is associated with an increase in the target/dependent variable of 0.6 units on average. This is true whatever the $R^2$ value is. But the 'on average' of course is important and it would be reasonable for the $R^2$ to affect how you interpret the coefficients, especially if you intend to use the model for prediction. So if the model $R^2$ is low, you might reduce your expectations of the predictive power of the predictor. But it would be better if you paid attention to the standard error of the coefficient, which measures the uncertainty in the parameter. This uncertainty is likely to be high if the model $R^2$ is low, but it's a more direct estimate of uncertainty in that specific parameter than the model $R^2$, which is a measure of the whole model.
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.