5
$\begingroup$

For $n$ observations $X_1, \ldots, X_n$ of one sample following distribution $P$ chosen from population $ \mathcal{P}$, suppose that we observe $X_i = x_i \in \mathbb{R}$. Empirically, we may detect that $P$ is not normal by looking at QQ-plot or run normality tests. When we have reasons to suspect that $P$ is not normal, empirically we may apply some sort of monotone transform (e.g. power transform, in particular Box-Cox transform) to "make it look more normal". This means we are going to fix a non-decreasing function $f: \mathbb{R} \to \mathbb{R}$ and work on $Y_i = f(X_i)$.

Is there any mathematical theory suggesting that, under suitably chosen $f$ (particularly the Box-Cox transform), the distribution of $Y_i$'s are going to, in some mathematical sense (e.g. distance between cumulative distribution functions), closer to a normal distribution with some mean and variance ? Quantitative answers(references) to this question are the best. Qualitative answers are also acceptable. I just want to know if such transformation is a purely empirical thing or does it have any theoretical fundation.

To me, it sounds like such transformations can only stretch the data (non-uniformly) so that it concentrates in some region than others, which should definitely make it closer to normal distribution. Whether for all non-normal distribution, one certain algorithm can determine a transform to make it more normal, is a different business. There is a lot of ambiguity in the use of such transforms.

Improve this question
$\endgroup$
5
  • 1
    $\begingroup$ Most monotone transformations address either skew, or non-constant variance, or both. Obviously no monotone transformation can convert any arbitrarily distributed variable to normal (take bimodal for example). $\endgroup$
    – Frans Rodenburg
    Commented Aug 10 at 16:25
  • $\begingroup$ Non-normal can of course never be made normal. But is there any mathematical result stating whether it will be closer in some sense? $\endgroup$
    – 温泽海
    Commented Aug 10 at 16:26
  • 4
    $\begingroup$ @Frans re "obviously:" On the contrary, any absolutely continuous random variable can be transformed via some monotone transform into a Normal variable. Simply apply the probability integral transform. $\endgroup$
    – whuber ♦
    Commented Aug 10 at 17:11
  • $\begingroup$ @whuber Fair enough, but that would require you know the CDF (or at least, can get a good empirical estimate of it) $\endgroup$
    – Frans Rodenburg
    Commented Aug 10 at 17:13
  • 2
    $\begingroup$ @Frans Of course!. One could attempt an estimate in any standard way--parametric, a KDE, or otherwise. But because that would have little or no value in most applications, and IMHO would tend to mislead the analyst, it's not worth pursuing in any detail. $\endgroup$
    – whuber ♦
    Commented Aug 10 at 17:25

1 Answer 1

Reset to default
6
$\begingroup$

It is often pointless to estimate such a transformation: extremely few statistical procedures require the underlying distribution to be Normal and any estimate will be imprecise anyway. But, as a mathematical proposition, when the underlying variable (call it $X$) has an absolutely continuous distribution $F$ (its cumulative distribution function), then there exists at least one increasing monotonic transformation $f$ for which $f(X)$ has the standard Normal distribution (whose CDF is $\Phi$). One such solution is

$$f = \Phi^{-1}\circ F;$$

that is,

$$Z = \Phi^{-1}(F(X))$$ has a standard Normal distribution.

This is a variant of the Probability Integral Transform.

Proof

The monotonicity of $f$ is self-evident.

Let $z$ be any number. Then

$$\Pr(Z\le z) = \Pr(\Phi^{-1}(F(X)\le z) = \Pr(F(X) \le \Phi(z)) = \Phi(z))$$

because $F(X)$ has a uniform distribution.. This is what it means for $Z$ to be standard Normal, QED.

Conversely, because no transformation can eliminate an atom from a distribution (that is, a point with positive probability), clearly non-continuous distributions cannot be transformed to Normal distributions.

Finally, there is an extensive theoretical foundation to the Box-Cox family and related families of transformations. Search our site for explanations and references.

Improve this answer
$\endgroup$
5
  • 1
    $\begingroup$ I failed to consider that this is still a monotone transformation... Which is funny, because only hours ago I posted a practical application of this in copula Gaussian graphical models. Another example where this is very useful is in the residual diagnostics of GLMs and other non-normal error structures. Here it is even extended to discrete probability distributions by randomly sampling between discrete values on the scale of the uniform. $\endgroup$
    – Frans Rodenburg
    Commented Aug 10 at 17:32
  • $\begingroup$ Thank you for the remark. However, this raises another question. Why doesn't people use this when they want normality, with $F$ replaced by the empirical cumulative distribution function? $\endgroup$
    – 温泽海
    Commented Aug 10 at 17:37
  • 1
    $\begingroup$ Because most of the time when people want normality they don't need it -- and that becomes an obstacle to a good analysis. Another reason is that statistics based on $f(X)$ are often uninterpretable: they bear very complicated relationships to properties of $X$ itself. $\endgroup$
    – whuber ♦
    Commented Aug 10 at 17:37
  • $\begingroup$ Thank you. I also have the major question. If the more used transforms (like Box-Cox) do not bring things directly to normality, why do we use them ? Do they bring the distribution closer to normality? $\endgroup$
    – 温泽海
    Commented Aug 10 at 18:43
  • $\begingroup$ Two principal reasons: to make the data more symmetric and to create homogeneous variances. Concerning the latter, see posts discussing spread vs. level plots. I posted a detailed example at stats.stackexchange.com/a/74594/919. $\endgroup$
    – whuber ♦
    Commented Aug 10 at 19:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.