# F and t statistics in a regression

In a multiple linear regression, why is it possible to have a highly significant F statistic (p<.001) but have very high p-values on all the regressor's t tests?

In my model, there are 10 regressors. One has a p-value of 0.1 and the rest are above 0.9

For dealing with this problem see the follow-up question.

-
See also here: how can a regression be significant yet all predictors be non-significant, & for a discussion of the opposite case, see here: significant t-test vs non-significant F-statistic. –  gung Sep 13 '12 at 15:15
add comment

## 3 Answers

As Rob mentions, this occurs when you have highly correlated variables. The standard example I use is predicting weight from shoe size. You can predict weight equally well with the right or left shoe size. But together it doesn't work out.

Brief simulation example

RSS = 3:10 #Right shoe size
LSS = rnorm(RSS, RSS, 0.1) #Left shoe size - similar to RSS
cor(LSS, RSS) #correlation ~ 0.99

weights = 120 + rnorm(RSS, 10*RSS, 10)

##Fit a joint model
m = lm(weights ~ LSS + RSS)

##F-value is very small, but neither LSS or RSS are significant
summary(m)

##Fitting RSS or LSS separately gives a significant result.
summary(lm(weights ~ LSS))

-
+1:> faily good way to make it intuitive. –  user603 Oct 13 '10 at 13:17
(+1) Very nice example! –  chl Oct 13 '10 at 13:25
It is interesting and important to note that both of your models predict equally well, in this case. High correlations among predictors are not necessarily a problem for prediction. Multicolinearity is only a problem when 1) analysts try to inappropriately interpret multiple regression coefficients; 2) the model is not estimable; and 3) SEs are inflated and coefficients are unstable. –  Brett Magill Jun 9 '11 at 14:27
add comment

This happens when the predictors are highly correlated. Imagine a situation where there are only two predictors with very high correlation. Individually, they both also correlate closely with the response variable. Consequently, the F-test has a low p-value (it is saying that the predictors together are highly significant in explaining the variation in the response variable). But the t-test for each predictor has a high p-value because after allowing for the effect of the other predictor there is not much left to explain.

-
add comment

A keyword to search for would be "collinearity" or "multicollinearity". This can be detected using diagnostics like Variance Inflation Factors (VIFs) or methods as described inthe textbook "Regression Diagnostics: Identifying Influential Data and Sources of Collinearity" by Belsley, Kuh and Welsch. VIFs are much easier to understand, but they can't deal with collinearity involving the intercept (i.e., predictors that are almost constant by themselves or in a linear combination) - conversely, the BKW diagnostics are far less intuitive but can deal with collinearity involving the intercept.

-
add comment