What does it mean for a linear regression to be statistically significant but has very low r squared?

Question

I understand it to mean that the model is bad at predicting individual data points but has established a firm trend (e.g. y goes up when x goes up).

Answer 1

It means that you can explain a small portion of the variance in the data. For instance, you can establish that a college degree impacts salaries, but at the same time it's just a small factor. There are many other factors that impact your salary, and the contribution of the college degree is very small, but detectable.

In practical terms it could mean that in average the college degree increases the salary by \$500 per year, while the standard deviation of salaries of people is \$10K. So, many college educated people have lower salaries than non-educated, and the value of your model for prediction is low.

Answer 2

It means "irreducible error is high", i.e., the best thing we can do (with linear model) is limited. For example, the following data set:

data=rbind(
cbind(1,1:400),
cbind(2,200:400),
cbind(3,300:400))
plot(data)

Note, the trick in this data set is that given one $x$ value, there are too many different $y$ values, that we cannot make a good prediction to satisfy all of them. At the same time, there are "strong" linear correlations between $x$ and $y$. If we fit a linear model, we will get significant coefficients, but low R squared.

fit=lm(data[,2]~data[,1])
summary(fit)
abline(fit)

Call:
lm(formula = data[, 2] ~ data[, 1])

Residuals:
     Min       1Q   Median       3Q      Max 
-203.331  -59.647   -1.252   68.103  195.669 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept)  123.910      8.428   14.70   <2e-16 ***
data[, 1]     80.421      4.858   16.56   <2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 93.9 on 700 degrees of freedom
Multiple R-squared:  0.2814,    Adjusted R-squared:  0.2804 
F-statistic: 274.1 on 1 and 700 DF,  p-value: < 2.2e-16

Answer 3

Put in a simple way (oversimplifying a bit) to prove that something is significant you need a strong effect and/or a lot of data. You may get a statistically significant linear regression even in the case of a small effect (small $R^2$) if you have enough data. This is not restricted to linear regression.

Answer 4

What does it mean for a linear regression to be statistically significant but has very low r squared?

It means that there is a linear relationship between the independent and dependent variable, but that this relationship might not be worth talking about.

The meaningfulness of the relationship, however, is very much contingent upon what you are examining but generally, you can take it to mean that statistical significance should not be confused with relevance.

With a large enough sample size, even the most trivial of relationships can be found to be statistically significant.

Answer 5

Another way of phrasing this is that it means you can confidently predict a change at the population level but not at the individual level. i.e. there is a high variance in individual data, but when a large enough sample is used, an underlying effect can be seen overall. It is one reason why some Government health advice is unhelpful to the individual. Governments sometime feel the need to act because they can see that more of some activity leads to more deaths overall in the population. They produce advice or a policy that 'saves' these lives. However, because of the high variance in individual responses, an individual may be very unlikely to personally see any benefit (or, worse, because of specific genetic conditions, their own health would actually have improved from obeying the opposite advice, but this is hidden in the population aggregation). If the individual derives benefit (e.g. pleasure) from the 'unhealthy' activity, following the advice may mean they forgo this definite pleasure throughout their lifetime, yet does not actually personally change whether they would or would not have suffered from the condition.