I have a large data of both binary, categorical and continuous variables. I would like to do the lasso to select the variables for regression. Besides, i also want to do pca to reduce the dimension. However, i was advised to do the standardization before lasso. But do I need to standardize the binary, categorical variables? If yes, which is the fastest way(in stata) to do the standardization for more than 200 varibles? Please help me with this. Thank you very nuch!
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
You should use your knowledge of the subject matter and your intended use of your model to decide whether and how to standardize your binary and multi-level categorical predictors.
In both PCA and LASSO, standardization helps to control for differences in scales of measurement so that all predictors are incorporated "fairly" in some sense into the model. PCA is fundamentally based on (scale-dependent) variances, and standard LASSO puts the same penalty parameter on the magnitudes of all regression coefficients (each of which is expressed in terms of the scale of measurement of the associated predictor). Thus the incorporation of an unstandardized continuous predictor in PCA or its penalization in LASSO would be different if a predictor based on length, say, were measured in kilometers rather than in millimeters. Standardization to unit standard deviation thus provides useful control for measurement scales with a continuous predictor variable.
It's not so clear how to put binary or multi-level categorical predictors onto comparable scales for incorporation into PCA or penalization in LASSO. Even with a binary predictor, scaling to unit standard deviation will depend on the prevalences of the two categories. Is that what you want in terms of treating all binary predictors "fairly" either with respect to each other or with respect to the continuous predictors?
With a multiple-level categorical predictor, trying to standardize the usual dummy-variable coding (multiple 0/1 predictors associated with the multiple-level categorical predictor) will lead to different standardizations depending on the choice of the reference level. And in LASSO it's quite possible that only one of the multiple levels will be incorporated into the final model unless you use a software routine that explicitly keeps all levels of a categorical predictor in the model. Is that what you want? You need to think about that issue carefully.
There is some further discussion of standardization of categorical predictors on this site. This page has some general discussion with links to further information. This answer goes into more detail about tradeoffs in standardizing categorical predictors for penalized methods like LASSO.
Furthermore, note that although PCA can reduce dimension in an important sense, it doesn't necessarily remove any of the predictor variables from the model as all of them might contribute to any of the retained principal components. LASSO by itself does dimension reduction by eliminating some predictors entirely from the model. So there is no need to do both, as your question might seem to indicate is your plan.
Finally, questions specific to how to accomplish a task in a particular software package like Stata are no on-topic here. A quick web check indicates that the Stata help list has some information on how to standardize predictor variables with that package.
