I am working with a dataset that involves a collection of one-hot encoded, ordinal, and numerical features. I am using a LASSO model. As the difference in scales can influence the estimates, I am standardizing the features. This transformation is obvious for numerical and ordinal features. But, for one-hot encoded features, I am running into a small issue. These set of features are stored as a sparse matrix and I remember reading somewhere (can't recall the source, probably Hands-on Machine Learning) that it is better to simply divide the feature by standard deviation without centering the data first to avoid breaking the efficient storage of sparse matrices. I am trying to understand the differences between this approach and the usual standardization. Why is this approach reasonable? Is it because the expected value of the underlying distribution (Bernoulli) is p (<1), so mean centering would make a huge difference to the scale?

Edit: Another follow-up question: would we even need to divide one-hot features by std. deviation, considering that the values already are either 0 or 1?



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.