I'm working on a regression problem. The dependant variable is skewed and has a distribution as below

enter image description here

I'm applying the log transformation but the resulting data is also skewed and is like below.

enter image description here

What should I do to resolve the skewness problem.

I'm using a tree-based model (e.g. Random Forrest). Is data skewness also a problem in tree-based models? How about the features? I believe having skewed data in tree based models is not an issue. Right?

  • 4
    $\begingroup$ As is reiterated throughout the site regression variables (dependent & independent) do not need to be normally distributed. The normality assumption is of the distribution of the error. $\endgroup$ – Alexis Jan 15 '19 at 2:48
  • $\begingroup$ so I don't need to transform any of the dependant or independent variables then? $\endgroup$ – HHH Jan 15 '19 at 3:01
  • 1
    $\begingroup$ One way to look at this is to look at a histogram of the residuals of your regression model. Are they normally distributed? Even if they aren’t, linear regression might still be robust to this. However, as I mentioned below, a Poisson or negative-binomial mode might work better just because it will make more sensible predictions: only integers, nothing below zero, etc. The assumptions of those models are more a priori reasonable than those of an ordinary least squares regression. $\endgroup$ – Mark White Jan 15 '19 at 3:05
  1. Are your data integers? It looks like you might have count data. In that case, look into Poisson and negative-binomial regression. These techniques are generally used for count data.

  2. A skewed dependent variable is not necessarily a problem for tree-based models per se—there are no assumptions in a decision tree that specify a conditional distribution of the errors, like in the generalized linear model.

However, what is the goal of your analysis? If you are looking for some type of inferential statistic or readily interpretable parameter, a random forest will not give you those. In that case, stick with a regression model.

I would suggest looking into the generalized linear model. In this, different link functions (kinda like transformations) are used to model data that cannot be negative, must be between 0 and 1, etc.

  • $\begingroup$ Answer to your 1st question, are you talking about the dependant variable? It's integer but not count (it's number of days in fact). In this case, what transformation should I use for the dependant variable? $\endgroup$ – HHH Jan 15 '19 at 2:51
  • 1
    $\begingroup$ Dependent variable, yes. Number of days is a count variable: it’s a count of the number of days. $\endgroup$ – Mark White Jan 15 '19 at 2:52
  • $\begingroup$ so should I use Poisson or negative-binomial regression instead of tree-based model? what is the advantage of those models? $\endgroup$ – HHH Jan 15 '19 at 2:57
  • 2
    $\begingroup$ It depends on the goals of your analysis. If you are trying to make predictions to future data, then a random-forest might perform better. But if you need interpretable parameters and inferential statistics, then a regression is what you need. As stated above, linear regression doesn’t assume that the dependent variable is distributed normally, but your data are going to violate the normal assumption because values can’t be below zero, and typical linear regression doesn’t appreciate that. A model for count data will. $\endgroup$ – Mark White Jan 15 '19 at 3:03
  • $\begingroup$ yes I'm trying to make prediction for the future instances $\endgroup$ – HHH Jan 15 '19 at 3:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.