# Why does adding an addition predictor to a linear regression mask correlation of other predictors to the response?

I'm using a multiple linear regression to determine the relationship between certain lifestyle characteristics of a species and the species's tolerance to stress.

The purpose of this analysis is simply to identify factors that may be correlated to differences in tolerance between species and not to predict tolerance (LT50).

Using three lifestyle characteristics (Microhabitat, Tidal Height and Motility) the model is adequate to predict the tolerance of animals (LT50). You can see that each characteristic is a significant model parameter:

lm(formula = LT50 ~ Tidal.Height + Microhabitat + Motility, data = juvdes)

Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept)   -7.0461     2.7913  -2.524   0.1276
Tidal.Height   5.5866     0.9304   6.004   0.0266 *
Microhabitat  10.2704     1.4577   7.046   0.0196 *
Motility      -6.2125     1.1151  -5.571   0.0307 *

Residual standard error: 2.129 on 2 degrees of freedom
Multiple R-squared:  0.9884,    Adjusted R-squared:  0.971
F-statistic: 56.78 on 3 and 2 DF,  p-value: 0.01736


When I add Weight as a variable, all other predictors become non-significant to the model:

 lm(formula = LT50 ~ Tidal.Height + Microhabitat + Motility +
Weight, data = juvdes)

Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept)     -0.4811    24.1076  -0.020    0.987
Tidal.Height     4.7692     3.2240   1.479    0.378
Microhabitat    11.4959     4.8678   2.362    0.255
Motility        -7.8169     6.0131  -1.300    0.417
Weight       -1580.6541  5731.5419  -0.276    0.829

Residual standard error: 2.903 on 1 degrees of freedom
Multiple R-squared:  0.9892,    Adjusted R-squared:  0.9461
F-statistic: 22.93 on 4 and 1 DF,  p-value: 0.1552


Why does adding weight as a predictors result in all other predictors becoming non-significant? Weight plays a significant role on tolerance (due to SA:V ratio) when comparing tolerance between different ages of a single species, so intuitively it should have some correlation with tolerance when comparing species.

• Something is strange. Weight is not statistically significant either. It has a very large negative value and an even larger standard error. I think there could be some form of collinearity introduced when weight is included. – Michael R. Chernick May 3 '17 at 21:15
• I think you're right. These data only cover 6 species. Tidal Height, Microhabitat and Motility are rankings based on the amount of stress encountered due to a certain characteristic (eg. motile low stress, low ranking and then sessile, high stress, high ranking). It turns out three species that share the same microhabitat also happen to be smaller than the other three by an order of magnitude. – hamilthj May 3 '17 at 21:19
• I believe this may be answered at stats.stackexchange.com/questions/3549: check it out. stats.stackexchange.com/questions/34488 is also relevant. Here you have far too few data to fit this model: you have almost as many parameters as observations! – whuber May 3 '17 at 21:36

@Gijs is correct, something is up with the weight variable. The massive standard error for that variable is a huge red flag. Have you looked at any collinearity diagnostics for the model? Check the condition number, if it's in around 30 or higher that indicates a serious collinearity problem. If your stats package can decompose the matrix you can see which correlations among the predictors are driving the collinearity issue (weight, I'm sure will be one of them).

Other diagnostics are tolerance (and VIF which is just the reciprocal of the tolerance). A simple approach is to look at a correlation matrix for your predictors, strong correlations, say in excess of .7, indicate collinearity.

As to why this is, how big is your sample? With a tiny sample there just may not be enough datapoints to handle the number of parameters you're trying to estimate.

Another possibility is that one of the other predictors is partly or entirely a function of the weight variable. This would lead to a strong correlation between weight and that variable and create collinearity. Is motility, for example, a function of the animal's weight?

The interesting thing too is that it's the constant that really takes a huge hit with the standard error when weight is added. Do you have the variables mean centered? Mean centering will shift any collinearity onto the constant term (idiots often propose mean centering as a solution for collinearity but it doesn't solve, it just hides it in the intercept).

In the bigger picture, the problem is that the model is trying to estimate the effect of each predictor as if it was unrelated to the other predictors. It is, in effect, throwing out the variance in any one predictor that is explained by the other predictors in the model. If one or more predictors is strongly related to another predictor then you end up tossing out almost all the variance in that predictor and so there's nothing really left to be used to estimate the effect of that predictor on the outcome. The model is essentially unsure what effect, if any, that variable has on the outcome. This imprecision is reflected in the large standard error.

One way to think about it is this, suppose height and weight were very strongly positively related to each other. If you used them both to predict some outcome then the model can't tell if it's height that matters or weight because tall people are heavy and short people are light. The model must essentially capitalize on tall people who are unusually light or short people who are unusually heavy. Especially in a small dataset there may not be many such people, hence increasing sample size usually helps with collinearity.

• I think the problem lies in the sample size and the collinearity between Microhabitat and Weight likely doesn't have any ecological significance. Maybe a better approach to suggesting relationships between factors would be pairwise product-moment correlations. Great answer. – hamilthj May 3 '17 at 22:26

The p-value you look at is a test that tells you how likely this coefficient estimate is, under the null-hypothesis. That null-hypothesis is that this coefficient is actually zero (but all the other components are still there).

If there is correlation between the predictors, it becomes very hard to estimate the coefficients. You can see that happening in the estimate for the coefficient for Weight, which is -1580, with a standard error of 5731. That means the model has no idea what the influence of weight is.

The main problem seems to be with the Weight column. Do you have measurements for different weights? That would make it hard to realiably estimate the influence of it.

• "If there is correlation between the predictors..." Do you mean here that if the predictors are covarying? Why does the model have no idea what the estimate of weight is? Weight is a measurement of the mean weight of animals used in the experiments. ySo all of species A weighed x, species b weighed y etc. – hamilthj May 3 '17 at 21:14
• Because there is collinearity between the columns, and / or, there is little variance in weight measurements. Imagine only a single measurement for weight. Then there are a lot of lines that fit through it, all equally well. – Gijs May 3 '17 at 21:18
• That's a good way of putting it. The collinearity is evident in the data, although sample size is also very small. – hamilthj May 3 '17 at 22:28