Should variables be dropped according to its skewness values? I am creating a classification model to predict the credit score of a person based on lots of factors. I got the dataset from kaggle. When I started doing the EDA part, I noticed that the skewness values of the numerical variables are too high such as 11, 20, etc. So is it ideal that I should drop those variables or is there any other option by which I can reduce the skewness and make it a normal distribution.
I tried checking the outliers of those numerical variables and each one of them contained outliers. The data contains 100000 samples with 28 variables.
 A: In general, skewness of covariates isn't a huge issue on its own. If you're using a tree-based model (like random forest, XGBoost, etc), you should not do anything about the skewness because at best, any transformations you do will do nothing, and in the worst case, transformations you make could hurt model performance on a test set.
If you're using a parametric model (like linear regression), you should check to see if the relationship between the skewed covariates and the outcome is linear by making a scatter plot and seeing if the points tend to follow a straight line. If this is false, you could try using some transformation to reduce skewness (like taking the log of a covariate) to see if it improves your model.
If you're just getting started in machine learning, I would recommend trying a bunch of different things to handle the skewness and seeing how that affects model performance.
A: Typically, when using a parametric model (e.g., regression), researchers use transformation to reduce skewness. These different transformations may include 1) log transform 2) square root transform and 3) box-cox transform. The purpose of these transformations is to make the dataset follow the Gaussian distribution - allowing the transformed dataset to be analyzed with the largest set of statistical tools (e.g., ANOVA, t-test, f-test, etc.).
Source:
https://www.kaggle.com/getting-started/110134
