# Can the bias introduced by lasso change the sign of a coefficient?

L1 penalized regression introduces a bias on your regression model but decreases the variance. When this bias is introduced, is it possible that the coefficient of $B$ changes sign? This would greatly affect the interpretation, where a regular regression might tell you an increase in $x$ increases $y$ by $B$, but maybe using lasso an increase in $x$ decreases $y$ by $B$

• I don't think so; if you compute the LASSO path from the least squares estimates to some point with high shrinkage, if a coefficient is shrunk so much that it would switch sign, at the point at which the sign change takes place, it is removed from the model (i.e. fixed at 0). This is in contrast to ridge regression where shrinkage can switch signs IIRC. – Gavin Simpson Jan 18 '13 at 20:26
• @Gavin That sounds good, but what if the coefficient that is removed from the model is for an interaction? It seems possible that removing it could then affect the signs of the underlying variables in almost any conceivable way. – whuber Jan 18 '13 at 21:47
• @whuber I guess that is just one of the problems with automated methods. I note that there are now grouped lasso methods and related techniques that work on groups of predictors, possibly in a hierarchy, that appear to attack this problem. – Gavin Simpson Jan 18 '13 at 22:11
• Isn't it important to consider also the size of the effect? If the coefficient is very close to 0 (on a scale that makes substantive sense) then do we care if the sign is positive or negative? – Peter Flom Jan 19 '13 at 15:45
• @PiotrSokol I just meant that the scale on which the IV was measured was sensible. E.g., for "weight of adult humans" pounds or kg makes sense. Tons does not make sense. In addition, some scales are purely arbitrary. – Peter Flom Apr 24 '13 at 22:55