Timeline for How to deal with high correlation among predictors in multiple regression?
Current License: CC BY-SA 4.0
16 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Sep 19, 2021 at 1:56 | history | edited | kjetil b halvorsen♦ | CC BY-SA 4.0 |
added 23 characters in body
|
| Jun 23, 2020 at 13:04 | comment | added | Peter Flom | Since I use SAS, I find their documentation particularly helpful. See here It also has references at the end. | |
| Jun 22, 2020 at 16:48 | comment | added | altabq | Could you point me in the direction of resources regarding your suggestion (3), combining variables, e.g., using partial least squares? | |
| Oct 14, 2016 at 11:20 | comment | added | Peter Flom | It's not necessary because, if there are a large number of variables, all pairs can be only slightly correlated yet the sum of them is perfectly colinear. It's not sufficient because there are cases where fairly high correlation does not yield troublesome collinearity per condition indexes | |
| Oct 14, 2016 at 7:27 | comment | added | Funkwecker | @Peter Flom: Why is correlation neither a necessary nor a sufficient condition for collinearity? Are you referring to non-linear correlation? | |
| Oct 20, 2013 at 0:49 | history | edited | Scortchi♦ | CC BY-SA 3.0 |
fixed typos
|
| Sep 29, 2012 at 15:13 | comment | added | Ander | I know stepwise is not very good, just used as an extreme confirmation possibility to see if the same predictors remain in the model. Usually I perform hierarchical, this analysis required only enter. I performed ridge regression and the beta weights have changed, increased for the rest of the predictors and are significant. The question is if this a good enough method for a report .. | |
| Sep 28, 2012 at 10:03 | comment | added | Peter Flom | It's a relatively technical book, but it probably would be helpful. In your situation you need to use a lot fewer predictors. Otherwise, results will be messy. Stepwise by the way, is not a good method of variable selection. | |
| Sep 28, 2012 at 5:07 | comment | added | Ander | Thank you for the answers! The Condition Index is ~ 29 with all the controls in and ~23 without them (5 variables). I conducted stepwise regression, the same 2 highly correlated variables remained the single significant predictors of the outcome. I do not understand if the partial correlations which are high for each of these variables matter as an explanation for why I have kept them in the model . Would Regression diagnostic: identifying influential data and sources of collinearity / David A. Belsley, Edwin Kuh and Roy E. Welsch, 1980" be helpful in understanding multicolinearity? | |
| Sep 27, 2012 at 15:29 | comment | added | Michael R. Chernick | I am 65. So before my time goes pretty far back. I got my PhD in 1978 but actually started using statistics in my first full-time job in 1969. | |
| Sep 27, 2012 at 13:27 | comment | added | Peter Flom | Hi @MichaelChernick. I am 53. Belsley wrote "the book" on collinearity. [amazon.com/…, Kuh and Welsch) covers collinearity but it is an earlier book by just [amazon.com/… that I used for my dissertation, which I started working on in 1996 or so. I don't know if Cook did anything on colinerity or not | |
| Sep 27, 2012 at 13:22 | comment | added | Michael R. Chernick | Peter has given you even more reason to trust his answer. I did not know he was an expert in multicollinearity. | |
| Sep 27, 2012 at 13:21 | comment | added | Michael R. Chernick | I assume that you are as old as me and therefore your work came after the work of Belsley, Kuh and Welsch and Cook. I know Cook's work was mostly on other diagnostic issues (leverage and non-normality), but did he do anything on multicollinearity? Of course the concept of ridge regression even goes back before my time | |
| Sep 27, 2012 at 10:56 | comment | added | Peter Flom | Thanks @MichaelChernick . I actually wrote my dissertation on collinearity diagnostics for multiple regression. | |
| Sep 27, 2012 at 10:54 | comment | added | Michael R. Chernick | Tabachnick and Fidell wrote a nice multivariate book for social science. They are not statististicians but their knowledge of multivariate is preety good. But I think they may create rules of thumb to simplify and could miss statistical subtleties. So I would rely more on what Peter says in his answers than in thier paper. | |
| Sep 27, 2012 at 10:15 | history | answered | Peter Flom | CC BY-SA 3.0 |