From what I understand about the data, there should not be a Y-intercept in the model (assume that for any regression model we obtain from mlr,l1,l2, the intercept is of negligible value). I understand that MLR doesnt care about the distribution of the data - one can even fit it without standardizing anything, whereas for L1/L2, the value matters - hence the question.

For example, let's say we have 2 features X1,X2, both with values between [-1,1]:

  1. X1 distribution: 90% of values lie greater than 0.8 ie num(x > 0.8) + num(x < -0.8) = 80% of data, with fewer points as we go further the tails

  2. X2 distribution: 90% lie within 0.2 ie [-0.2,0.2]

Standardizing by standard deviation would cause X2 to have extreme values at the tails. Standardizing by Max of abs(X) (for this dataset means not standardizing since X1,X2 is already [-1,1]) would cause X1 to heavily penalised if in the "wrong" direction as compared to Y.

Should the solution be:

  1. Standardize X1 by std & Standardize X2 by abs value?
  2. Still standardize by std

I am concerned because incorrectly choosing the standardization procedure would like cause the features to be unfairly weighted in the Model - L1 would mean wrongly choosing features, L2 would mean incorrectly weighted.

Let me know your thoughts or correct me if Im wrong. Thanks in advance!


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.