1
$\begingroup$

I'm working on a paper and I have some problems explaining some of my findings. I have four independent variables, let's call them A,B,C,D. And i have one dependent variable, let's call it Y.

In my correlation analysis I have these correlations between the dependent variable (Y) and the four independent (A,B,C and D). Y -- A=0.54*, B=0.29* C=0.44* D=-.08

I then did a multiple regression analysis, where i used the four independent variables to predict the dependent variable. My model was significant, and with a high F value. But my Standardized coefficient estimates between the variables are different. Y-- A=.40* B=.08 C=.22* D=-.13*

As you can see D is now a significant predictor of Y, and at the same time B is not. I have checked for multicollinearity, and all looks good (to the best of my knowledge). I understand that there most be some underlying relationship between the independent variables, but how do you find it and explain it?

Note: I have read some similar questions here, but I don't find them covering the same question I have.

$\endgroup$

2 Answers 2

4
$\begingroup$

It appears that your predictors share a particular type of multi-colinearity known as suppression. Suppression refers to a pattern of multi-colinearity that results in a larger or reversed effect in multiple regression compared to simple regression. Since the partial correlation of D is larger (and sig.) in the multiple regression case than in the simple regression case we know that the unique effect of D that is not shared with the other predictors was being "suppressed" in the simple regression case.

Remember that whereas simple regression presents a "total" effect of your single predictor, multiple regression yields the "unique" effect of each predictor after partialing out any shared explanatory power. For example, if we wanted to estimate children's weights from their heights and ages, height and age likely explain a lot of the same variability in weight because taller children tend to be older.

To illustrate suppression, let's say we wanted to predict children's weights using age and number of hours involved in organized sports to predict weight. What we might imagine is that older children tend to be involved in more sports (i.e. age and # hours are positively correlated). We might also reason that children who are involved in more organized sports tend to weigh less than children who are not involved in organized sports. However, since age and involvement in organized sports are positively correlated, some part of the # hours-weight relationship is really driven by the relationship between age and weight (which is positive). Since both this positive correlation and negative correlation are going into the overall correlation between # hours and weight, the total effect may appear smaller than the partial effect. Here is a visual representation of this:

enter image description here

Each circle represents one of the variables in the example. The dotted line within the two predictor circles separates out the portion of that predictor that shares explanatory power with the other predictor. The small portion of the # hours circle is the portion of explanatory power that is is shared with age. The dotted lines represent the portion of the total effect of a predictor that is really driven by the other predictor. For example, the dotted line with the + symbol is the part of the # hours simple relationship that is really driven by child age. The solid lines represent the partial correlation of that predictor with the outcome over and above any shared explanatory power of the two predictors. The total effect would be a combination of the dotted and solid lines stemming from the same predictor variable. As we can see, the + and - relationships would sort of wash each other out. Thus, the partial correlation would be stronger than the simple correlation. The way that suppression works with more than 2 predictors gets a bit more complicated, but I bet there is some overall relationship among your predictors. If you calculate (or ask the computer to calculate) tolerance, this should help you know what's going on. Tolerance varies from 0 to 1 and tells you the extent to which your predictors are orthogonal. 1 indidicates that there is no multicollinearity and 0 indicates that there is complete multicolinearity (fun fact, if you regressed D on A, B and C the $R^2$ you would obtain would be 1-tolerance). Since there are more than 2 predictors, the multi collinearity issue is more complex than just, "are there simple relationships among my predictors". As such, I think tolerance would be a better measure of this. In terms of explaining why such suppression exists, I would think about the simple example above and consider whether it would make sense that such a phenomenon is happening with your own data. Hope this is helpful!

A tolerance of .67 means that something like 33% of the variability in one of your predictors is explained by the other 3 predictors. This seems, to me, like a non trivial amount of redundancy.

$\endgroup$
1
  • $\begingroup$ Thank's, that helped a lot. (i will give you +1 when i have reached 15 reputation) I'm looking at the relationship between jobcrafting (four dimentions) and work engagement, so i would guess there most bee some realtionship between the crafting dimentions (that's not a big surprice). It's just that i haven't found any similar problem in any article about the realtionship between jobbcrafting and work engagement. $\endgroup$
    – Flave
    Commented May 8, 2015 at 9:43
1
$\begingroup$

Are the results that you show for the multiple regression really correlations? or are they the coefficient estimates?

With your check on multicolinearity, what did you see? was there some relationship, just not enough to worry you? or was there really no relationship?

Remember that correlations and simple linear regressions are interpreted very differently from a multiple regression and even moderate relationships between predictor (independent) variables can lead to very different results in coefficient estimates and significance. If Y is strongly related to the difference between say A and D, then it could be that the relationship between Y and D is not significant without A in the model, but with both A and D in the model, that difference is now part of the model and D becomes significant. Additionally if B is somewhat related to that difference then you may see that when only B is in the model, but with both A and D in the model, B becomes redundant to the difference and is no longer significant.

Here is a simple example in R that shows with moderate correlation, neither x1 or x2 alone is a significant predictor of y, but including both is very significant:

set.seed(1)
x1 <- rnorm(25)
x2 <- rnorm(25, x1)
y <- x1-x2 + rnorm(25)
pairs( cbind(y,x1,x2) )
cor( cbind(y,x1,x2) )
summary(lm(y~x1))
summary(lm(y~x2))
summary(lm(y~x1+x2))
$\endgroup$
1
  • $\begingroup$ As you say, they are Standardized coefficient estimates. When i checked on multicolinarity I checked the tolerance level (it's between 0.67 and 0.97) and VIF (it's between 1,02 and 1,48). I also checked the visual parts, Homoscedasticity and Normality of Errors. There may be some relationship when i see the Collinearity Diagnostics between B and D, but not much. $\endgroup$
    – Flave
    Commented May 7, 2015 at 17:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.