While building predictive models we often see skewness in the target variable. Then we generally take transformations to make it more normal. We generally do it for linear models and not for tree based models. This actually means that our distribution is not normal, we are deliberately making it normal for prediction. Why do we do that? What is the advantage we get when outcome is normal?
4
-
1$\begingroup$ Please note that (approximate) lack of skewness is not the same as (approximate) normality. The point is that symmetric distributions are more economical to describe. It is unrealistic in most applications to expect a symmetrized distribution to be normal. $\endgroup$– whuber ♦Commented Feb 3, 2017 at 14:50
-
$\begingroup$ So why do we do transformations before training linear models ? $\endgroup$– ArchitCommented Feb 4, 2017 at 6:36
-
1$\begingroup$ There are many reasons. Some are best appreciated by looking at examples, such as the analysis I describe at stats.stackexchange.com/a/74594/919. $\endgroup$– whuber ♦Commented Feb 4, 2017 at 16:11
-
$\begingroup$ Frequently it's not necessary to do so (it's the characteristics of the conditional distribution that matter more than the marginal), and there can several possible reasons, but it's often done for misplaced reasons. Can you give a context where this was done, so we might try to figure out why it was done there? $\endgroup$– Glen_bCommented Feb 5, 2017 at 23:37
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
My belief that this is because of a common misconception about a normality assumption in linear regression.
When we make an assumption about normality in linear regression (related to maximum likelihood estimation), it is about the error term, not about the marginal/pooled distribution of $y$. However, many people do not know this, just look at $y$, and apply a transformation to try to make $y$ more normal.
Tree-based models make no normality assumption, so people do not feel compelled to transform $y$, even though they probably did not need to do a transformation in the linear model, either.
If you have a theoretical reason to care about predicting a transformation of $y$, then it would make sense to transform $y$ for any predictive model, whether linear, tree-based, or otherwise.
