I'm not sure I understand stepwise multiple regression, so I'll first try to share my understanding of it: We use several IV to predict one DV. In forward selection, we first enter the IV that increases $R^2$ the most. Then we enter the IV that increases $R^2$ the second most etc., and we do so for every variable that increases it significantly (or maybe we use AIC or BIC as criteria? however, that part isn't important to me right now).

In backward selection we enter all IVs into the model and removes the weakest one, as long is the decrease in $R^2$ isn't significant (or whatever criteria we're using).

I can see how these methods would yield different results if the variables were correlated/colinear. In this situation, two IVs would predict the same variance in the DV twice, which is why the order the IVs are entered matter.

Here's the part I find confusing. In regression do we not assume that the variables are not colinear? I've been made to understand that this is why collinearity is a problem. However, when explaining how individual IVs are assessed my lecturer described that regression uses the residuals that are left when controlling for other IVs. That is, we first enter IV $A$ and then we use IV $B_{residual}$ to predict $DV_{residual}$. If that's how multiple regression works haven't we already accounted for different IVs predicting the same variance? I can see how this would mean that colinearity matters, but if there is no collinearity the order in which the variance is predicted should not matter.

Perhaps I've misunderstood either colinearity or what entering another predictor into a regression model means. I would be happy if someone could help me understand this, or at least point me in the right direction.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.