I have a dataset with 10000 data points and 20 features. The features are not normally distributed (most of them have a generalized extreme value or burr distribution and all values are greater or equal to zero). Of course some classifiers requires standardized/normalized features so that the features have similar scale. Because I have some outliers in my data, I think I have to do standardization (and not normalization). Currently I'm subtracting the mean of each feature and dividing by the standard deviation. Another option would be to subtract the median and divide by the IQR.

Which of this two options is better or does it not matter?

  • $\begingroup$ Before scaling your data, have you attempted a transformation of your features to get them normal (I assume that would make your analysis easier) ? $\endgroup$ – Tommaso Guerrini Apr 5 '16 at 13:38
  • $\begingroup$ @TommasoGuerrini Yes, I have tried the box-cox transformation which works well for most features but it does not work for some features (which have lot of zeroes). For those features I have just added a small constant but the transformation still does not work well. $\endgroup$ – machinery Apr 5 '16 at 16:04
  • $\begingroup$ Substracting the mean and division by standard deviation will not change the shape of the distribution and will not make it normal. If you really need to transform data to Z-scores, I would recommend to use Box-Cox wherever it is possible and just discard all features that have a lot of zeroes. You can not make them "normal", of course. $\endgroup$ – German Demidov Apr 5 '16 at 16:32
  • $\begingroup$ @GermanDemidov I don't want to make the features normal. Of course, Naive Bayes and Logistic Regression assumes normal distributed features (right?), so for these type of classifiers I will do BoxCox transformation. My intention with subtracting the mean and dividing by the standard deviation is to make the features to have a similar scale, independent of the distribution. Is this not right? $\endgroup$ – machinery Apr 5 '16 at 18:05
  • $\begingroup$ Logistic regression does not assume normally-distributed features, so no need to transform for that, at least not because of the lack of normality. Naive Bayes doesn't either, but it assumes independent features, so again, no need to transform. (Of course if you are using a Naive Bayes calculation package, the package itself may assume Normal features, but that's not inherent in N.B. itself.) $\endgroup$ – jbowman Apr 5 '16 at 20:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.