0
$\begingroup$

The dependent variable seems to resist to become normal. Here I have depicted the probplot of the variable in three situations. The top plot depicts the original data, the middle plot shows log1p(y) and the last one highlights BoxCox transformed data.
Are there any other transformations that would work better?
Transformations

$\endgroup$
  • $\begingroup$ Why do you think your dependent variable should be normal? What kind of analysis are you using it in? $\endgroup$ – Glen_b Jul 31 '18 at 4:25
  • $\begingroup$ @Glen_b It's a regression problem and so I expect that by making my Y normal I would gain better results. $\endgroup$ – Reza Jul 31 '18 at 7:23
  • $\begingroup$ There's no assumption that the marginal distribution of Y should be normal in regression. (i.e. the thing you're looking at has no need whatever to look normal, it's unrelated to any regression assumption). This is addressed by answers to many questions on our site. When doing the usual testing or confidence intervals or prediction intervals there is an assumption that relates to normality but the thing you're looking at is of almost no use for that assumption (because the assumption relates to the error term or equivalently the conditional distribution of Y, not its marginal distribution). $\endgroup$ – Glen_b Jul 31 '18 at 11:32
1
$\begingroup$

The problem you are facing is a common one! It can be solved by a clever analyst, but is really viewed as more art than science.

I will point you to a fantastic answer found at this link where the OP was asking a similar (though less detailed) question. You, however, already understand the basics of variable transformation to make your residuals approximately normal.

For other more 'exotic' transformations, you may be interested in the link found here, which describes alternative data transformation methods. Look for the section which starts with the following:

Third piece of advice. If the error variances are unequal, try "stabilizing the variance" by transforming y

This page details transformations you may be looking for to coerce your residuals to normality, including arcsine transformation and reciprocal transformation.

Without your actual dataset to work with, one can not recommend a specific method that would definitively "work better" than your current methods, as we can not test our guesses using trial-and-error.

$\endgroup$
  • $\begingroup$ The Johnson Distribution could also be applied as a transformation. See: Youn-Min Chou, Alan M. Polansky & Robert L. Mason (1998) Transforming Non-Normal Data to Normality in Statistical Process Control, Journal of Quality Technology, 30:2, 133-141, DOI: 10.1080/00224065.1998.11979832 Control Charts for Non-Normal Data: Illustrative Example from the Construction Industry Business wseas.us/e-library/conferences/2014/Malaysia/MACMESE/… $\endgroup$ – blackeneth Jul 30 '18 at 20:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.