4
$\begingroup$

I know that the penalized Regression is robust to collinearity. I also found a page at scikit-learn : Plot Ridge coefficients as a function of the regularization says it "shows the effect of collinearity in the coefficients of an estimator."

Here is what beats me. There are pattern in heavy wave, or in mild slope; some variables are in same trend, others not.
How do I observe the effect of collinearity from the regularization path plot?
Also does L1 and L2 regularizer has different pattern for collinear variables?
Or detect collinearity using path plot is bad idea?

$\endgroup$
1
$\begingroup$

I would say that change of sign for regression coefficient as alpha goes from K to zero is such indicator.

EDIT:

Huge standard errors and change of signs when omitting/adding variables is a symptom of problems caused by multicollinearity. It is almost impossible to estimate marginal effect from the variables having these problems when they are so strongly related.

I would say this path plot can show which are those variables to which you have to put attention.

| cite | improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ Can you expand on your answer? $\endgroup$ – Glen_b Mar 3 '15 at 11:10
  • $\begingroup$ Added more explation on the issue $\endgroup$ – Analyst Mar 3 '15 at 13:39
  • 1
    $\begingroup$ yes, but the L2 path do nothing with add/omitting variables. Or are you talking about LASSO path? the large change at kink is hint of collinearity? $\endgroup$ – WeiChing 林煒清 Mar 4 '15 at 4:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.