Just wondering -- I just ran a regression where my $R^{2}$ value was 0.046 but I have four independent variables with p-values below 0.05. So, at first glance I thought I had a coherent model when really that was nowhere near the case.

Why exactly does this happen? Is there an intuitive explanation for this apparent paradox?

  • 3
    $\begingroup$ There is no paradox here. You are able to detect some structure in the distribution of your response variable but there is too much unexplained variance. Especially if you have a relatively "biggish" sample that is a common phenomenon (the "big sample makes everything significant" rant). eg. N = 500; set.seed(9); x1 = rep(c(1,1.5), each=0.5*N); y = x1 + rnorm(N,sd=2); summary(lm(y~x1)). "Lights are on ($p$ value $< 1e-5$) but nobody's home ($R^2 < 0.05$)"; $R^2$ and $p$-values are generally bad indicators of model coherency. $\endgroup$ – usεr11852 Mar 31 '17 at 23:19

Let's use @user1182's simple linear regression example. Your scenario is an extension on the higher dimensional space.

enter image description here

            Estimate Std. Error t value Pr(>|t|)    
(Intercept)  -0.8791     0.4384  -2.005   0.0455 *  
x1            1.6846     0.3439   4.898 1.31e-06 ***

The regression line has a significant regression slope. This is not surprising because the line is indeed trending upward. However, the coefficient only makes sense if the OLS assumptions are met. The assumptions are not met, the linear model can't explain the data thus $R^2$ is low.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.