# Mathematical Theory of Monotone Transforms

For $$n$$ observations $$X_1, \ldots, X_n$$ of one sample following distribution $$P$$ chosen from population $$\mathcal{P}$$, suppose that we observe $$X_i = x_i \in \mathbb{R}$$. Empirically, we may detect that $$P$$ is not normal by looking at QQ-plot or run normality tests. When we have reasons to suspect that $$P$$ is not normal, empirically we may apply some sort of monotone transform (e.g. power transform, in particular Box-Cox transform) to "make it look more normal". This means we are going to fix a non-decreasing function $$f: \mathbb{R} \to \mathbb{R}$$ and work on $$Y_i = f(X_i)$$.

Is there any mathematical theory suggesting that, under suitably chosen $$f$$ (particularly the Box-Cox transform), the distribution of $$Y_i$$'s are going to, in some mathematical sense (e.g. distance between cumulative distribution functions), closer to a normal distribution with some mean and variance ? Quantitative answers(references) to this question are the best. Qualitative answers are also acceptable. I just want to know if such transformation is a purely empirical thing or does it have any theoretical fundation.

To me, it sounds like such transformations can only stretch the data (non-uniformly) so that it concentrates in some region than others, which should definitely make it closer to normal distribution. Whether for all non-normal distribution, one certain algorithm can determine a transform to make it more normal, is a different business. There is a lot of ambiguity in the use of such transforms.

• Most monotone transformations address either skew, or non-constant variance, or both. Obviously no monotone transformation can convert any arbitrarily distributed variable to normal (take bimodal for example). Commented Aug 10 at 16:25
• Non-normal can of course never be made normal. But is there any mathematical result stating whether it will be closer in some sense?
– 温泽海
Commented Aug 10 at 16:26
• @Frans re "obviously:" On the contrary, any absolutely continuous random variable can be transformed via some monotone transform into a Normal variable. Simply apply the probability integral transform.
– whuber
Commented Aug 10 at 17:11
• @whuber Fair enough, but that would require you know the CDF (or at least, can get a good empirical estimate of it) Commented Aug 10 at 17:13
• @Frans Of course!. One could attempt an estimate in any standard way--parametric, a KDE, or otherwise. But because that would have little or no value in most applications, and IMHO would tend to mislead the analyst, it's not worth pursuing in any detail.
– whuber
Commented Aug 10 at 17:25

It is often pointless to estimate such a transformation: extremely few statistical procedures require the underlying distribution to be Normal and any estimate will be imprecise anyway. But, as a mathematical proposition, when the underlying variable (call it $$X$$) has an absolutely continuous distribution $$F$$ (its cumulative distribution function), then there exists at least one increasing monotonic transformation $$f$$ for which $$f(X)$$ has the standard Normal distribution (whose CDF is $$\Phi$$). One such solution is

$$f = \Phi^{-1}\circ F;$$

that is,

$$Z = \Phi^{-1}(F(X))$$ has a standard Normal distribution.

This is a variant of the Probability Integral Transform.

Proof

The monotonicity of $$f$$ is self-evident.

Let $$z$$ be any number. Then

$$\Pr(Z\le z) = \Pr(\Phi^{-1}(F(X)\le z) = \Pr(F(X) \le \Phi(z)) = \Phi(z))$$

because $$F(X)$$ has a uniform distribution.. This is what it means for $$Z$$ to be standard Normal, QED.

Conversely, because no transformation can eliminate an atom from a distribution (that is, a point with positive probability), clearly non-continuous distributions cannot be transformed to Normal distributions.

Finally, there is an extensive theoretical foundation to the Box-Cox family and related families of transformations. Search our site for explanations and references.

• I failed to consider that this is still a monotone transformation... Which is funny, because only hours ago I posted a practical application of this in copula Gaussian graphical models. Another example where this is very useful is in the residual diagnostics of GLMs and other non-normal error structures. Here it is even extended to discrete probability distributions by randomly sampling between discrete values on the scale of the uniform. Commented Aug 10 at 17:32
• Thank you for the remark. However, this raises another question. Why doesn't people use this when they want normality, with $F$ replaced by the empirical cumulative distribution function?
– 温泽海
Commented Aug 10 at 17:37
• Because most of the time when people want normality they don't need it -- and that becomes an obstacle to a good analysis. Another reason is that statistics based on $f(X)$ are often uninterpretable: they bear very complicated relationships to properties of $X$ itself.
– whuber
Commented Aug 10 at 17:37
• Thank you. I also have the major question. If the more used transforms (like Box-Cox) do not bring things directly to normality, why do we use them ? Do they bring the distribution closer to normality?
– 温泽海
Commented Aug 10 at 18:43
• Two principal reasons: to make the data more symmetric and to create homogeneous variances. Concerning the latter, see posts discussing spread vs. level plots. I posted a detailed example at stats.stackexchange.com/a/74594/919.
– whuber
Commented Aug 10 at 19:15