10
$\begingroup$

My dataset ($N \approx 10,000$) has a dependent variable (DV), five independent "baseline" variables (P1, P2, P3, P4, P5) and one independent variable of interest (Q).

I have run OLS linear regressions for the following two models:

DV ~ 1 + P1 + P2 + P3 + P4 + P5
                                  -> R-squared = 0.125

DV ~ 1 + P1 + P2 + P3 + P4 + P5 + Q
                                  -> R-squared = 0.124

I.e., adding the predictor Q has decreased the amount of variance explained in the linear model. As far as I understand, this shouldn't happen.

To be clear, these are R-squared values and not adjusted R-squared values.

I've verified the R-squared values using Jasp and Python's statsmodels.

Is there any reason I could be seeing this phenomenon? Perhaps something relating to the OLS method?

Improve this question
$\endgroup$
6
  • 1
    $\begingroup$ numerical issues? The numbers are quite close to each other... $\endgroup$
    – user166243
    Commented Dec 4, 2017 at 15:20
  • $\begingroup$ @user2137591 This is what I'm thinking, but I have no idea how to verify this. The absolute difference in R-squared values is 0.000513569, which is small, but not that small. $\endgroup$
    – Cai
    Commented Dec 4, 2017 at 15:21
  • 1
    $\begingroup$ I hope you know linear algebra: if $\mathbf{X}$ is the design matrix of the above, could you please compute $\det\mathbf{X}^{T}\mathbf{X}$, where $T$ is the matrix transpose and $\det$ is the matrix determinant? $\endgroup$
    – Clarinetist
    Commented Dec 4, 2017 at 15:25
  • 9
    $\begingroup$ Missing values get auto-dropped? $\endgroup$
    – generic_user
    Commented Dec 4, 2017 at 15:29
  • 1
    $\begingroup$ 0.000513569 is a very small number: it is 0.41 percent change. It is very possibly a numerical issue. What Clarinetist is trying to say is that maybe your design matrix has a poor condition number and when inverting it is numerically instable... $\endgroup$
    – user166243
    Commented Dec 4, 2017 at 15:32

1 Answer 1

Reset to default
27
$\begingroup$

Could it be that you have missing values in Q that are getting auto-dropped? That'd have implications on the sample, making the two regressions not comparable.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.