How does a variable's distribution inform decisions on predicting it with regression analysis?

Question

I was reading a forum post on Kaggle here. The person is showing an example of exploratory analysis for a competition.

During the exploratory analysis one of the charts generated is a log(targetVariable) distribution. Here is what it looks like:

I generated the same chart based on not log of the data just for my interest:

ggplot(full) + geom_histogram(mapping=aes(x=loss))

And it looks like this:

So what? Now what? Why do this? Two facts have been learned:

1. The target variable is skewed left (Is that correct?) (It is not correct, see comment below)
2. The log of the target variable is normally distributed

In the context of multiple linear regression, how does this inform my decisions on prediction? Do I have to log() all my predictor variables now? Why? Other than generating a nice shiny chart, how does exploratory analysis of the distribution of thew target variable help when it comes to predicting the target variable?

Answer 1

In this case, the y variable has a noticeable positive skew. It's hard to tell from the relatively low-resolution figure of the original variable, but it's possible that there is an abrupt cutoff just to the left of the mode.

This might be an indication of a variable that has an artificial left truncation. The "true" value of an employee to a workplace, for example, might range from high positive values to even negative values. If you try to ascertain this from paid wages, though, you will encounter an artificial left cutoff in the form of a minimum wage. Taking a logarithm of this type of variable can make it more normally distributed (as in the case you show). Say you're considering the effect of number of school years and recorded felonies on wages. The true model might be one that includes values that have been artificially been truncated.

This, however, leads to the question of whether taking the logarithm will improve the regression's performance. Saying that an increase in $x$ will cause a linear increase in $y$, is very different than saying that an increase in $x$ will cause a linear increase in $log(y)$ - these are contradictions, in fact.

Linear Regression Models with Logarithmic Transformations describes the four conceptual possibilities of taking logarithms in regression (a logarithm can be taken on a dependent and/or independent variable).

I think the important point is that the decision of whether to take a logarithm of the dependent variable depends not only on it, but also on the dependent variable(s). For one, as the paper above explains, it really depends on what the true model is. Also, note that, to some extent, linear regression assumes that the residual is normally distributed, and the residual depends on transformations to all variables.

So while the graphs you show are some indication that a log transformation on the y might be beneficial (it makes it more normal), you can't fully decide that based on the y alone.