Are skewed distributions problematic to binary classification models? I am trying to build a binary classification model and while trying to build the model, I have done some research about skewed distributions. I learnt that skewed models should be normalized to be tested for a proper statistical test, but I am not sure what effects skewed distributions have to the binary classification models.
And also, while doing my research on building binary classification models, most people I referenced tried data standardization as well as normalization for their skewed data distributions, but I don't know what benefits standardization and normalization will bring to building a binary classification model. I understand that we do standardization and normalization to suppress big numbers dominating the feature space hoping that those standardization and normalization can even bring skewed distributions to normal distributions, for example, log-transformation.
But I don't really have a firm understanding about the connection between skewness and normalization with a binary classification model but I kind of understand if I link the skewness with statistical tests. But when I try to think of the skewness in my dataset with my binary classification model, I cannot really get the clear picture what problems the skewness will bring for my binary classification model.
Notice that I am not talking about the class imbalanced dataset but I am only talking about distributions of features.
 A: The effect of skewness will differ between different classification methods.
Decision-tree methods are insensitive to the scaling of the predictors. Some suggestions for methods that are invariant with respect to monotonous transformations (e.g., log transform, squaring) of the predictors:

*

*Single decision trees based on the CART or MOB algorithms, which are implemented in the rpart and partykit R packages. The advantage of the latter is that it does not have a preference towards splitting on variables with a higher number of possible split points. Package partykit also implements the conditional inference tree algorithm, which is a good method, but it employs linear association tests for variable selection, which may be more sensitive to the scaling of the predictors.


*Decision-tree ensembles based on the previously mentioned tree algorithms, as implemented in the randomForest and mobForest packages. Or a CART-based gradient boosting decision-tree ensemble based on the previously mentioned tree algorithms, as implemented in package gbm.


*Prediction rule ensembles (PREs) based on MOB or CART. PREs aims to strike a balance between the ease of interpretability of a single tree, and the higher predictive accuracy of a decision-tree ensemble. This method is implemented in package pre (disclaimer: which I developed with Benjamin Christoffersen). Specify argument use.grad = FALSE in order to derive the rules using MOB, specify argument tree.unbiased=FALSE to derive the rules using CART (otherwise, default is to use conditional inference trees).
Normalizing or standardizing are linear transformations and will not mitigate skew:
set.seed(42)
skew <- 100*rbeta(100000, 2, 6) # generate skewed data
breaks <- seq(min(skew), max(skew), length.out = 20)
hist(skew, main = "Skewed raw", breaks = breaks)
norm <- (skew-min(skew)) / (max(skew)-min(skew)) # normalizes
breaks <- seq(min(norm), max(norm), length.out = 20)
hist(norm, main = "Normalized", breaks = breaks)
stan <- scale(skew) # standardizes
breaks <- seq(min(stan), max(stan), length.out = 20)
hist(stan, main = "Standardized", breaks = breaks)


