# Why increasing Lasso regression alpha value the root mean squared error only increases?

Hi i am fitting a Lasso model using different values in the range of 2*10^-5 to 500 for the alpha parameters like:

alphas=np.linspace(0.00002,500,20)


when i plot the negative root mean squared error and absolute error from cross validation i get a graph like this:

so the error increases in modulo instead of decreasing... why am i getting this result?

• By definition of "least squares," the RMS error must be least when $\alpha=0$ because it is directly proportional to the sum of the squares of the error--which is precisely the quantity being minimized.
– whuber
Commented Dec 9, 2020 at 22:17
• yes but from what i understand for lasso in this case since the graph is the error i get from cross validation, it should decrease in the module up to a certain alpha value and then increase again or i am wrong? Commented Dec 9, 2020 at 22:22
• Were you expecting it to decrease? Larger $\alpha$ means stronger penalization and less model freedom, hence increased RMS error. And an aside: you'll probably see more interesting behavior setting $\alpha$ less than one. The algorithm may be scaling the data so that larger values result in a completely empty model. Commented Dec 9, 2020 at 22:24
• For some examples of what these plots typically look like, see stats.stackexchange.com/questions/319861 and stats.stackexchange.com/questions/488675. The latter looks like it might ask the same question you are.
– whuber
Commented Dec 9, 2020 at 22:26
• I just saw your comment--I think the result will make more sense using a range of smaller $\alpha$ values. It looks to me like you overshot the the decrease. Commented Dec 9, 2020 at 22:27