9
$\begingroup$

My dataset ($N \approx 10,000$) has a dependent variable (DV), five independent "baseline" variables (P1, P2, P3, P4, P5) and one independent variable of interest (Q).

I have run OLS linear regressions for the following two models:

DV ~ 1 + P1 + P2 + P3 + P4 + P5
                                  -> R-squared = 0.125

DV ~ 1 + P1 + P2 + P3 + P4 + P5 + Q
                                  -> R-squared = 0.124

I.e., adding the predictor Q has decreased the amount of variance explained in the linear model. As far as I understand, this shouldn't happen.

To be clear, these are R-squared values and not adjusted R-squared values.

I've verified the R-squared values using Jasp and Python's statsmodels.

Is there any reason I could be seeing this phenomenon? Perhaps something relating to the OLS method?

$\endgroup$
  • 1
    $\begingroup$ numerical issues? The numbers are quite close to each other... $\endgroup$ – user166243 Dec 4 '17 at 15:20
  • $\begingroup$ @user2137591 This is what I'm thinking, but I have no idea how to verify this. The absolute difference in R-squared values is 0.000513569, which is small, but not that small. $\endgroup$ – Cai Dec 4 '17 at 15:21
  • 1
    $\begingroup$ I hope you know linear algebra: if $\mathbf{X}$ is the design matrix of the above, could you please compute $\det\mathbf{X}^{T}\mathbf{X}$, where $T$ is the matrix transpose and $\det$ is the matrix determinant? $\endgroup$ – Clarinetist Dec 4 '17 at 15:25
  • 8
    $\begingroup$ Missing values get auto-dropped? $\endgroup$ – generic_user Dec 4 '17 at 15:29
  • 1
    $\begingroup$ 0.000513569 is a very small number: it is 0.41 percent change. It is very possibly a numerical issue. What Clarinetist is trying to say is that maybe your design matrix has a poor condition number and when inverting it is numerically instable... $\endgroup$ – user166243 Dec 4 '17 at 15:32
23
$\begingroup$

Could it be that you have missing values in Q that are getting auto-dropped? That'd have implications on the sample, making the two regressions not comparable.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.