I am using a GBM for a regression machine learning problem. I believe one of the assumptions for GBM and most ML algorithms (at least for regression problems) is that the y-variables are normally distributed. a histogram of my y-variables look like this:

distribution of my data

If it helps, I can provide the dput() of this vector as well. What are some good transformation functions I can use on my data to have it be more gaussian? and what are some good ways (other than looking at a histogram) to see if the transformation is a good one?



You can use BoxCox transformation for converting skewed distribution to more Gaussian-like. A code excerpt:

from scipy.stats import boxcox, skew

xt, _ = boxcox(x)
x_skew = skew(x)
xt_skew = skew(xt)

You will notice a lower skewness for the transformed data.

| cite | improve this answer | |
  • $\begingroup$ Box-Cox often won’t play if any values are negative. $\endgroup$ – Nick Cox Jan 25 at 16:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.