10
$\begingroup$

I have this confusion about which transformation to use in my data. The histogram of my original data looks like this

enter image description here

Now I have seen at most of the places to take log transformation in case the data is positively skewed. But when I take log transformation I get something like this which is negatively skewed, not what I desire.

enter image description here

If I take square root transformation and cube root transformation I get like this

enter image description here enter image description here

Now the data is pretty much close to normal. But I didn't get the intuition behind this. Why didn't it work with the log transformation when I see at so many places people mentioning about log transformation useful when data is positively skewed. Here square root and cube root worked.

I want to know at which condition we should use log transformation and at which condition to use cube root transformation. Suggestions?

Improve this question
$\endgroup$
1
  • 4
    $\begingroup$ The short answer may seem unhelpful, but it is whatever works and is valid, so long as it helps your goals. Assuming all your values are positive, then all three transformations you've tried are well defined and it's clear that cube root works best if it is important that you have a symmetric distribution. $\endgroup$
    – Nick Cox
    Commented Aug 15, 2013 at 10:06

1 Answer 1

Reset to default
10
$\begingroup$

It's not clear from your question why you need to transform at all. (What are you trying to achieve and why?)

As for why logs might make the appearance more symmetric in some cases and not others, not all distributions are the same - while log transformations may sometimes make skewed data nearly symmetric, there's no guarantee that it always does.

Often other transformations do much better.

For example logs work very nicely on lognormal distributions, while cube roots do better on gamma. Below, $a$ is simulated from a lognormal distribution, and $b$ from a gamma distribution. They look vaguely similar, but the log-transform makes $a$ symmetric (in fact, normal), while making $b$ left-skewed. On the other hand a cube root transformation leaves $a$ still somewhat right skew, but makes $b$ very nearly symmetric (and pretty close to normal):

log vs cube root, lognormal vs gamma

Other times there's simply no monotonic transformation to achieve approximate symmetry (e.g. if your distribution is discrete and sufficiently skew, like a geometric(0.5), or say a Poisson(0.5), no monotonic transformation can make it reasonably normal - wherever you put them, the leftmost spike will always be taller than the next one).

Incidentally, you might want to use more bars on your histograms, and maybe consider using other displays as well, to get a handle on the distributional shape. See my cautionary tale.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Very neat answer. Would it make sense to do a skewness test on the log-transformed distribution and examine whether it is significantly lower than zero, thus a gamma GLM with log-link is preferred over simple OLS with log-transformed response? $\endgroup$
    – Jean-Paul
    Commented Feb 5, 2018 at 11:08
  • $\begingroup$ Not to me it wouldn't. ... but to explain why in any detail would be lengthier than a comment. this should be a new question. $\endgroup$
    – Glen_b
    Commented Jun 7, 2018 at 22:39
  • 1
    $\begingroup$ Briefly why looking at the skewness doesn't help: 1. consider a shifted lognormal. No matter what the shift parameter, the skewness is the same -- but the effect of a log transform on the shape is quite different (if you shift up a lot, taking logs does very little) 2. consider a lognormal with $σ=0.164$; the population skewness is just below $0.5\,$. Now consider a gamma with shape parameter $\alpha=1$. Its skewness is $2$ -- i.e.considerably higher skewness -- but you should still take logs in the first case and something less strong in the second case (between $x^{1/3}$ & $x^{1/4}$). $\endgroup$
    – Glen_b
    Commented Oct 13, 2019 at 6:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.