3
$\begingroup$

I've looked at some Kaggle notebooks lately of people using Lasso/Ridge for linear regression. The majority that I've seen don't seem to standardize the predictors before they fit Lasso/Ridge even though the variables are on disparate scales (e.g., multiple orders of magnitude in difference)

Here are a couple of Jupyter notebooks that I've seen that uses no standardization:

https://www.kaggle.com/mohaiminul101/car-price-prediction

https://www.kaggle.com/burhanykiyakoglu/predicting-house-prices/comments

Most of the notebooks I've seen actually lack this standardization, and I only look at the top rated notebooks for popular datasets, so I was thinking there their methodology may be more reputable. So now I'm wondering if there's something I'm missing, or if people are indeed being negligent/incorrect by not standardizing when using regularization.

Is there any theoretical justification or practical advantage to not performing standardization on regressors when they exist on disparate scales?

Improve this question
$\endgroup$
6
  • 2
    $\begingroup$ I assume these are just beginner notebooks rather than competition entries $\endgroup$
    – seanv507
    May 8, 2021 at 16:13
  • $\begingroup$ All serious regularised regression (LASSO/Ridge) I have read standardised their variables. Especially within Python, where we might often use pipeline, dropping a StandardScaler(somewhat mindlessly) is trivial to implement. $\endgroup$
    – usεr11852
    May 11, 2021 at 0:11
  • $\begingroup$ @usεr11852 Can you suggest a resource where I may be able to find more reputable linear regression data analyses? Not advanced analysis per se, just something that shows reputable beginning to end procedure $\endgroup$
    – student010101
    May 11, 2021 at 14:38
  • $\begingroup$ Kuhn & Johnson (2013) Applied Predictive Modeling is really readable and to the point. $\endgroup$
    – usεr11852
    May 11, 2021 at 21:23
  • $\begingroup$ @usεr11852 I'll check that out. Is it more focused on prediction rather than inference? $\endgroup$
    – student010101
    May 11, 2021 at 21:47

1 Answer 1

Reset to default
2
+25
$\begingroup$

The main reason I can think of is to penalize variables on different scales differently. However, unless different variables are deliberately re-scaled differently, I would rather guess that it's an oversight. See also this previous question.

Improve this answer
$\endgroup$
1
  • $\begingroup$ I agree with you, I would also add that many libraries standardize variables by default (e.g., glmnet), so maybe models in those notebooks do that, but it's just not done explicitly in the code (read the docs). Also, kaggle notebooks are not necessarily a pinnacle of best modeling practiecs $\endgroup$
    – rep_ho
    May 12, 2021 at 17:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.