14
$\begingroup$

When running ridge regression, how do you interpret coefficients that end up larger than their corresponding coefficients under least squares (for certain values of $\lambda$)? Isn't ridge regression supposed to monotonically shrink coefficients?

On a related note, how does one interpret a coefficient whose sign changes during ridge regression (i.e., the ridge trace crosses from negative to positive on a ridge trace plot)?

Improve this question
$\endgroup$
1
  • $\begingroup$ Ridge regression only monotonically shrinks coefficients in the case of an orthogonal design matrix. In the presence of correlations, its impossible to say anything of that generality. $\endgroup$
    – Matthew Drury
    Commented Nov 12, 2015 at 14:28

1 Answer 1

Reset to default
5
$\begingroup$

As $\lambda$ increases from zero the contribution of various coefficients changes to suit the optimization, allowing both value increases and sign changes. Have a look at Ryan Tibshirani's ridge regression charts (PDF) illustrating both of your questions (charts 17, 19).

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.