0
$\begingroup$

Hi i am fitting a Lasso model using different values in the range of 2*10^-5 to 500 for the alpha parameters like:

alphas=np.linspace(0.00002,500,20)

when i plot the negative root mean squared error and absolute error from cross validation i get a graph like this: enter image description here

so the error increases in modulo instead of decreasing... why am i getting this result?

Improve this question
$\endgroup$
7
  • $\begingroup$ By definition of "least squares," the RMS error must be least when $\alpha=0$ because it is directly proportional to the sum of the squares of the error--which is precisely the quantity being minimized. $\endgroup$
    – whuber ♦
    Commented Dec 9, 2020 at 22:17
  • $\begingroup$ yes but from what i understand for lasso in this case since the graph is the error i get from cross validation, it should decrease in the module up to a certain alpha value and then increase again or i am wrong? $\endgroup$
    – Sunny
    Commented Dec 9, 2020 at 22:22
  • $\begingroup$ Were you expecting it to decrease? Larger $\alpha$ means stronger penalization and less model freedom, hence increased RMS error. And an aside: you'll probably see more interesting behavior setting $\alpha$ less than one. The algorithm may be scaling the data so that larger values result in a completely empty model. $\endgroup$
    – abstrusiosity
    Commented Dec 9, 2020 at 22:24
  • $\begingroup$ For some examples of what these plots typically look like, see stats.stackexchange.com/questions/319861 and stats.stackexchange.com/questions/488675. The latter looks like it might ask the same question you are. $\endgroup$
    – whuber ♦
    Commented Dec 9, 2020 at 22:26
  • 1
    $\begingroup$ I just saw your comment--I think the result will make more sense using a range of smaller $\alpha$ values. It looks to me like you overshot the the decrease. $\endgroup$
    – abstrusiosity
    Commented Dec 9, 2020 at 22:27

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.