4
$\begingroup$

I have linear regression model clustered on both firm and data. when I add two more variables to the regression model, one of the newly added variable is significant, but the R squared remains the same as the model without these two variables. Could this be the case? If so, why?

$\endgroup$
7
$\begingroup$

If you are adding new variables and not getting any change in $R^2$, the most likely explanation is that your new variables are linear combinations of the existing variables, so that your design matrix does not have full rank. I would recommend finding the rank of your design matrix, both before and after the addition of your variables. This can be done in R using the rank function, but you need to be careful with it, because rank calculations can be subject to tolerance issues in computation.

It is theoretically possible for new explanatory variables to add nothing to the $R^2$ of a regression, even if they are not linearly combinations of existing variables. However, this requires some exact vector angle conditions that are essentially impossible for random data. You can construct this by design, but it does not occur at random with continuous data (and even for discrete data it occurs with vanishingly small probability). For this reason, I strongly suspect that you are adding variables that are just linear combinations of your existing explanatory variables.

$\endgroup$
3
  • $\begingroup$ Hi Ben, thanks for your suggestion. I haven't looked at the rank of my independent variable matrix (will do that), but I did check the correlation matrix of all independent variable. And I do not detect very high correlations. Does that give some hints on whether the newly added variables are the linear combination of existing variables? $\endgroup$
    – Tracy Yang
    Jun 20 '18 at 13:47
  • $\begingroup$ Not necessarily - if a new variable is a linear combination of several others then it could have low correlation with each of them individually, so it is not necessarily a hint. A better test would be to regress your new variables against the old ones - if they are linear combinations, you will get a perfect fit (i.e., $R^2=1$). $\endgroup$
    – Ben
    Jun 20 '18 at 22:55
  • $\begingroup$ Under what conditions would the correlation of the new variable be low with the original variables but the multiple correlation high? Are you simply referring to low correlations with many variables or did you have something more in mind? $\endgroup$
    – Joel W.
    Jun 22 '18 at 13:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.