1
$\begingroup$

From what I understand about the data, there should not be a Y-intercept in the model (assume that for any regression model we obtain from mlr,l1,l2, the intercept is of negligible value). I understand that MLR doesnt care about the distribution of the data - one can even fit it without standardizing anything, whereas for L1/L2, the value matters - hence the question.

For example, let's say we have 2 features X1,X2, both with values between [-1,1]:

  1. X1 distribution: 90% of values lie greater than 0.8 ie num(x > 0.8) + num(x < -0.8) = 80% of data, with fewer points as we go further the tails

  2. X2 distribution: 90% lie within 0.2 ie [-0.2,0.2]

Standardizing by standard deviation would cause X2 to have extreme values at the tails. Standardizing by Max of abs(X) (for this dataset means not standardizing since X1,X2 is already [-1,1]) would cause X1 to heavily penalised if in the "wrong" direction as compared to Y.

Should the solution be:

  1. Standardize X1 by std & Standardize X2 by abs value?
  2. Still standardize by std

I am concerned because incorrectly choosing the standardization procedure would like cause the features to be unfairly weighted in the Model - L1 would mean wrongly choosing features, L2 would mean incorrectly weighted.

Let me know your thoughts or correct me if Im wrong. Thanks in advance!

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.