2
$\begingroup$

In regression, when 2 parameters are correlated and added to a model separately, how likely is it that one parameter will be a significant predictor of the response variable while the other is not? To me this seems unlikely but I've encountered it in a publication. Based on my understanding and experience of multi-collinearity, if one variable is highly correlated with another, both should be influential in a model. Under what conditions could this occur?

Context: For pedagogical purposes I am replicating an GLMM analysis from a 2004 Science paper that used observational data which was posted with the paper. They are interested in causal interpretations of 2 parameters that are correlated (Pearson's=0.67, p<0.01). They state that the 1st parameter is significant in their regression (beta=0.24,SE=0.05,p<0.01) while the other is not (b=0.02,SE=0.07,p=0.51). The language of the paper implies that they ran two separate models, one with each predictor on its own (but see edit below). A model with just the 1st predictor also has a lower AIC than a model with just the 2nd predictor (335.9 vs 341.0).

I can replicate the much of the analysis, including the correlation, the coefficients of the 1st parameter, and qualitatively the difference in AIC.

Beyond not being able to replicate their analysis (perhaps I am missing some modeling detail), I don't understand how they could have two correlated predictors yet end up with only one being significant in the model.

**EDIT:**The paper implies that they are reporting coefficients for two separate models that had only one or the other predictor. However, the coefficients that they report possibly came from a model that had both predictors in at the same time. I can now replicate the beta and SE for the 2nd parameter. They appear to have ignored issues of multi-colliearity in running and interpreting their models.

I am still wondering if my original intuition about their being a problem with the reported values is generally correct, or are their conditions where two correlated variables will behave differently in a fitted model if entered in separately.

$\endgroup$
  • 2
    $\begingroup$ Because significance depends on sample size as much as anything else, there is no universal relationship between correlation of regressors and significance of their coefficients. It is possible for two regressors to have arbitrarily high correlation (but still less than $1$, so they are not perfectly correlated) and for one to have a high p-value (above 50%) and the other to have an arbitrarily low p-value. $\endgroup$ – whuber Nov 11 '15 at 18:19
1
$\begingroup$

The effect of two predictors being correlated is to increase the uncertainty of each's contribution to the effect. For example, say that $Y$ increases with $X_1$, but $X_1$ and $X_2$ are correlated. Does $Y$ only appear to increase with $X_1$ because $Y$ actually increases with $X_2$ and $X_1$ correlated with $X_2$ (and vice versa)? The difficulty in teasing these apart is reflected in the width of the standard errors of your predictors. (The SE is a measure of the uncertainty of your estimate.)

We can determine how much wider the variance of your predictors' sampling distributions are as a result of the correlation by using the Variance Inflation Factor (VIF). For two variables, you just square their correlation, then compute:
$$ VIF = \frac{1}{1-r^2} $$ In your case the VIF is $2.23$, meaning that the SEs are $1.5$ times as wide. It is possible that this will make only one still significant, neither, or even that both are still significant, depending on how far the point estimate is from the null value and how wide the SE would have been without any correlation.

To get a stronger sense of how the individual contributions of correlated variables can appear different when both are included vs. not, it may help to read my answer here: Is there a difference between 'controlling for' and 'ignoring' other variables in multiple regression?

To see a demonstration showing that whether the variables are significant can vary, you may want to check out my answer here: How seriously should I consider the effects of multicollinearity in my regression model?

$\endgroup$
  • $\begingroup$ Thanks. This helps with my general understanding of multi-coll. Your answer seems to relate to when you have a model with two correlated predictors in it, eg y ~ x1 + x2. However, my main interest is when you run two models M1=y ~ x1 and M2=y ~ x2. If x1 and x2 are correlated, how likely will it be that x1 will have a significant association with y but x2 will not given that x1 and x2 are correlated. $\endgroup$ – N Brouwer Nov 11 '15 at 18:06
  • 1
    $\begingroup$ Perhaps I misunderstood, @NBrouwer. It depends on how several factors (eg, the correlation, but also the size of the true effect, N, etc) trade off. Your correlation, r = .67, is not really that strong 45% of the variance in 1 can be predicted using the other. It is certainly possible that a model w/ 1 would be significant & a model w/ the other wouldn't be. $\endgroup$ – gung - Reinstate Monica Nov 11 '15 at 19:24
  • $\begingroup$ To be clear about your main response, if the model y ~ x1 + x2 was fit and x1 and x2 have are correlated with VIF of 2.23, then the SE of each should be multiplied by 2.23? The author's main parameter of interest had beta=0.24 with SE=0.05 and p<0.01. The SE should therefore be 2.23*0.05=0.11. Constructing a simple 95% CI 1.96*0.11 = 0.23. The p value should therefore be about 0.06 not <0.01. Is this all accurate? Thanks for all the assistance. $\endgroup$ – N Brouwer Nov 11 '15 at 20:08
  • $\begingroup$ @NBrouwer, no. The variance has been computed correctly by the model. The variance is already 2.23 times larger than it would have been if the variables had been perfectly uncorrelated. The standard error is the square root of the variance (again, the increased width has already been incorporated correctly into the SE). $\endgroup$ – gung - Reinstate Monica Nov 11 '15 at 21:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.