5
$\begingroup$

Just before I start the question I would like you all to know that I have checked the other threads on taking the log of variables but I still think I have a question that hasn't been touched on yet. I would also like to thank whuber for his lengthy answer to another log question here.

This question specifically relates to one of the reasons of why we take logs, namely transforming the distribution of the data. When we take the log of a variable it is usually because the distribution of the variable is skewed and we want to give it a normal distribution. A common example of this in OLS regressions in economics is a variable denoting wages, income, GDP etc. However, no one ever seems to mention the central limit theorem (CLT). The CLT says that sum of many random variables will be normally distributed even if their underlying distributions are not normally distributed. If the error is the sum of the random variables $X$ and $Y$, $\epsilon = Y - X\beta$, then surely the error will be normally distributed regardless of the distribution of $X$ and $Y$. If this holds (and the CLT seems to hold under pretty weak conditions) then why would we need to transform the variable?

$\endgroup$
  • $\begingroup$ Whuber's exemplary answer does mention "making the distribution more symmetric". Not necessarily normal. But even if we did it in an attempt to induce normality, have you ever estimated a sample of infinite dimensions? No. Are you certain that "a few hundred" or "a few thousand" or even "a few tens of thousands of observations" are enough so that the effects of the CLT will emmerge? If you are, you shouldn't be. The one condition in the CLT that is not weak at all, is the requirement that the sample size goes to infinity. $\endgroup$ – Alecos Papadopoulos Nov 9 '13 at 2:38
  • $\begingroup$ OK well we'll deal with symmetric then. Why would we want to make the distribution of one of the variables symmetric if the CLT says it doesn't matter about the shape of the underlying distributions? With regards to $n$, I always though that the remarkable thing about the CLT was that it's effects are apparent even as you move from say $n = 5$ to $n = 20$. Obviously we can't estimate over infinite samples but isn't that a problem for any type of analysis that involves a probability limit?? $\endgroup$ – EconStats Nov 9 '13 at 2:59
  • $\begingroup$ People take logs in response to observing that the distribution is not normal. If the empirical distribution isn't normal looking, you can't really invoke the CLT to say that it is. As to why this isn't the case, one reason is that the underlying population might be heterogenous. $\endgroup$ – quasi Nov 9 '13 at 3:48
  • $\begingroup$ @EconStats I can see that you indeed have great faith in the CLT taking effect "very quickly". Experience has shown that this is not the case as often as we would all like, especially when we move to more sophisticated estimators. $\endgroup$ – Alecos Papadopoulos Nov 9 '13 at 6:27
  • $\begingroup$ "The CLT says that sum of many random variables will be normally distributed even if their underlying distributions are not normally distributed." --- well, not quite. It's not actually a theorem (or rather a collection of theorems) about sums, but about standardized averages (such as $\frac{\bar x-\mu}{\sigma/\sqrt n}$), and it doesn't apply to every possible distribution. $\endgroup$ – Glen_b Nov 9 '13 at 9:05
2
$\begingroup$

You might find this display interesting:

These are residuals from a linear regression with ten x-variables (IVs), a skewed error distribution (but one with all moments finite, to which the CLT definitely applies!), and 1000 observations (i.e. the data was simulated).

It's a normal qqplot, which if the residuals are close to normal should look reasonably close to a straight line.

normal qq plot for a skewed error distribution

Clearly, it's not remotely normal looking! The residuals are still pretty skewed.

Okay, maybe I didn't have enough variables. Here's one for 100 x-variables:

lognormal qq plot, p=100

The plot is very similar - and still very skew.

So with n=1000 and p=100, we're not seeing anything like what you say we should be seeing.

$\endgroup$
  • $\begingroup$ You're right I meant to say that the sum will tend to be normally distributed so I was sure (as you have detailed) there will be situations when it won't work. To bring this back to the original question, if you had taken the logs of those variables, would that have made a difference to the Q-Q plot? Or are these situations where the CLT fails so dire that no transformation can save them? $\endgroup$ – EconStats Nov 11 '13 at 19:41
  • $\begingroup$ I'm puzzled; this question seems to contradict the sense of the original, which was 'why transform when we have the CLT?'. Note that my example isn't some extreme case - I deleiberately chose one where the CLT applies. That's what you get when the CLT works. You seem to be currently asking "well if we transform, does the CLT do what I thought?". With continuous variables, monotonic transformation can give you normality directly, without relying on the CLT; for $Y|X \sim F_{Y|X}(y)$, $Y^* = \Phi^{-1}[F_{Y|X}(Y)]$ will be normal. ... (ctd) $\endgroup$ – Glen_b Nov 11 '13 at 21:38
  • $\begingroup$ (ctd)... without any averaging at all. On the other hand, if you have discrete distributions, it's possible to have situations where no transformation will give you even approximate normality. $\endgroup$ – Glen_b Nov 11 '13 at 21:41
  • 1
    $\begingroup$ I see it all the time ... I think sometimes it happens because people don't know (or don't fully absorb) the distinction. I've just been teaching from a text (not my choice of text) which described the distinction at length, and then went ahead later in the book and committed exactly the error it cautioned against in the early chapters, and did it over and over. [However, if you understand well how the $x$'s are distributed, you can sometimes get some idea of the conditional distribution even from an unconditional display, so it's not automatically a mistake -- but it very often is.] $\endgroup$ – Glen_b Nov 12 '13 at 2:51
  • 1
    $\begingroup$ I think that we should treat it as a mistake even when it isn't, because its so often misunderstood; it generates many questions here from people who transform things that are fine - and they often try to transform their $x$'s to normality too. For some discussion of the distinction, see here or the second example here. $\endgroup$ – Glen_b Nov 12 '13 at 2:56
0
$\begingroup$

Per your comment, Lindberg-Feller CLT requires independence (but not identically distributed), along with a finite means and variance. Are you sure that the "Y can't be [independent] by definition but this is the case for all regressions" part doesn't kill your argument? Just because it's true by definition doesn't mean that it's not true (or applicable).

$\endgroup$
  • $\begingroup$ Are you sure it's not applicable? This is a quote from Greene's Econometric Analysis "Since nearly all the estimators we construct in econometrics fall under the purview of the CLT, it is obviously an important result." As for iid, I know that for most econometric models it is the LF CLT which provides the definition I gave above. So because that works for unequal variances I don't know if they have to be identically distributed. They do have to be independent, which for OLS the $X$s have to be (the $Y$ can't be by definition but this is the case for all regressions) $\endgroup$ – EconStats Nov 9 '13 at 2:49
  • $\begingroup$ You're right, and I'll edit my answer. $\endgroup$ – Wayne Nov 9 '13 at 3:17
  • 1
    $\begingroup$ They don't have to be independent. CLT's exist for dependent sequences, like martingales and mixing processes. (cc @EconStats) $\endgroup$ – Alecos Papadopoulos Nov 9 '13 at 6:30
  • $\begingroup$ @AlecosPapadopoulos: Yes, but you can't mix-n-match requirements. EconStats is appealing to the Lindberg-Feller CLT, which does require independence. $\endgroup$ – Wayne Nov 11 '13 at 21:56
  • $\begingroup$ I didn't see a reference to L-F CLT in the OP's question. $\endgroup$ – Alecos Papadopoulos Nov 11 '13 at 22:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.