# Box-Cox transform doesn't make data normal

I'm working with the famous diamonds dataset and the target value is non-normal:

After applying the Box-Cox transform, the shape of the histogram is closer to a normal distribution but the quantile plot is far from it:

Also, all normality tests fail to reject non-normality with tiny p-values. This happens after I have filtered the outliers in my data using the IQR criterion.

Is there a way to fix the normality of my target variable? Why is the Box-Cox transform failing?

• You have included the regression tag. Linear regression makes no assumption that $y$ is normal. When we make a normality assumption in linear regression, the assumption is about the error term, not the pooled/marginal distribution of $y$. You might be transforming for no reason.
– Dave
Jul 11, 2022 at 11:40
• I would take a log transform because your data is positive and right skewed and then run my regression. If you want to use a model that assumes normality like OLS then you should check that assumption on the residuals not on the target variable. So as Dave said you might not need any transformation at all. Jul 11, 2022 at 11:50
• Related to both the two comments above; a log transformation of prices of diamonds may be useful for reasons other than conditional normality (which you can't assess from your plot, since you're looking at the marginal distribution, whose appearance may be non-normal when the assumptions hold). Such a transformation may well help improve linearity, heteroskedasticity and make the distribution around the conditional mean less skew, though my first thought may well have been to do something else than transform the data. Jul 12, 2022 at 3:26

## 1 Answer

Quick points about the issues you raised:

1. There is no guarantee that the Box-Cox transformation will transform your data to look more like a normal distribution.
2. Do not remove "outliers" because they do not fit your model, change your model.
3. Often times there is nothing you can do (no transformation) that can make a variable "normal".

More importantly:

1. Why are you trying to transform your variable into a normal distribution?
2. Why are you removing "outliers"?
3. Your data is bounded from below by 0 (or close to 0, so it appears), so there is no way to make this normal in any way, shape, or form.