5
$\begingroup$

It is well-known that when a linear transformation is applied to a normally-distributed random variable, the result is itself a normally-distributed random variable.

I am interested in the converse of this. If I apply an unknown transformation to a normally-distributed random variable, and find that the result is normally-distributed, does this imply that the transformation must be linear?

It is apparent that this cannot be true in the most general case, as one can invent pathological non-linear transformations that preserve the distribution while being non-linear, e.g. $$ x \to\begin{cases}-x & |x|<1\\x&\mathrm{otherwise}\end{cases} $$ will not affect a normal distribution centred on $x=0$. However, can the statement be rendered true by placing some (relatively weak) restrictions on the form of the transformation, e.g. that it is in some sense smooth and continuous?

$\endgroup$
  • 1
    $\begingroup$ Does stats.stackexchange.com/questions/200380/… answer your question? Or are you asking something different--and if so, how? $\endgroup$ – whuber Feb 15 '19 at 15:38
  • 1
    $\begingroup$ @whuber That question proves that there are non-affine transformations that preserve normality. My interest is in whether it is possible to use preservation of normality to infer that a transformation must be affine, if we place some weak restrictions on the character of that transformation (e.g. smoothness, somehow defined). I think your answer hits the mark. $\endgroup$ – avid Feb 19 '19 at 5:16
4
$\begingroup$

The thread at Normal Distribution Existence Non-affine Invariant Transformation? exhibits many non-affine transformations that map normally distributed variables into normally distributed variables, so the answer is in the negative.

(For the record, an affine transformation $f:\mathbb R\to \mathbb R$ is one of the form $f(x) = \mu + \sigma x$ for numbers $\mu$ and $\sigma.$ That is, its graph is a line. I will restrict this definition to positive $\sigma$ (positively sloping graphs) in order to obtain an easily stated result.)

We can, however, point to a simple condition that assures a positive answer, as suggested in the question. I will state it rather generally because that reveals its nature and it is no harder to prove the generalization.

Let $X$ be a continuous random variable supported on all the real numbers. (This implies its distribution function $F_X$ given by $F_X(x)=\Pr(X\le x)$ is a strictly increasing continuous function from $\mathbb R$ onto the interval $(0,1).$)

Denote by $\mathcal{F}[F_X]$ the set of distributions of all invertible affine transformations of $X$ (the "location-scale family" of $X$). Thus

$$\mathcal{F}[F_X] = \{F_{\mu+\sigma X}\mid \mu\in\mathbb{R},\,\sigma\gt 0\}.$$

Proposition

Let $f:\mathbb R\to\mathbb R$ be a continuous, increasing, one-to-one, measurable function for which $F_{f(X)}\in\mathcal{F}[F_X].$ Then $f$ is affine.

Proof

The measurability of $f$ is required to assure $Y=f(X)$ is a random variable.

By applying a preliminary affine transformation to $f(X)$ we may, without any loss of generality, assume $F_X=F_Y=F_{f(X)}.$

Let $y$ be any real number and compute

$$F_X(y) = F_Y(y) = F_{f(X)}(y) = \color{red}{\Pr(f(X) \le y) = \Pr\left(X\le f^{-1}(y)\right)} = F_X\left(f^{-1}(y)\right).$$

The first three and last equalities are all definitions. The remaining (penultimate) equality, shown in red, is justified because $f$ is one-to-one and increasing, whence its inverse $f^{-1}$ exists, is unique, and preserves inequalities.

Finally, comparing the left and right sides of the foregoing, observe that because $F_X$ is strictly increasing and continuous, equality can hold only when the arguments are equal: that is,

$$F_X(y) = F_X\left(f^{-1}(y)\right)\text{ implies } y = f^{-1}(y)$$

for all real $y.$ That means $f^{-1}$ (and therefore $f$) is the identity function. Accounting for the preliminary affine transformation that was applied, we conclude that the original transformation was affine, QED.

Readers might enjoy constructing counterexamples to the proposition when any of the conditions on $f$ are relaxed.


To answer the question: This applies to Normally distributed variables because they are a location-scale family of absolutely continuous variables with distributions supported on the entire real line.

$\endgroup$
  • $\begingroup$ Great answer! Can you explain where the assumption that $f$ is continuous is used in your Proposition? $\endgroup$ – Artem Mavrin Feb 18 '19 at 1:50
  • $\begingroup$ Also, I think your conditions may be relaxed to just “$f$ is strictly increasing” since strictly increasing functions on $\mathbb{R}$ are automatically Borel-measurable (as are continuous functions, if that assumption is needed) $\endgroup$ – Artem Mavrin Feb 18 '19 at 1:52
  • $\begingroup$ @Artem Thank you for pointing out those implications. The continuity is used to conclude $f$ is a surjection. Although it's not a necessary assumption here, I first introduced it in my considerations when pondering extensions to non-continuous random variables, and left it in. Obviously, though, if $f$ is not continuous, then $f(X)$ cannot be a continuous variable. $\endgroup$ – whuber Feb 18 '19 at 12:54
0
$\begingroup$

If $y=f(x), x,y\in\mathbb{R}^m$, then the condition that $x$ and $y$ are normally distributed is that the Jacobian of the transformation $f$ exists and is constant everywhere, with thanks to @henry for improvements.

Please see the comment by @whuber in the OP’s question, viz., Normal Distribution Existence Non-affine Invariant Transformation?”, for his detailed discussion of non-affine transformations that preserve normality.

$\endgroup$
  • 1
    $\begingroup$ You may need differentiability of $f$ for that statement (and something like continuity for the original statement) $\endgroup$ – Henry Feb 15 '19 at 8:40
  • $\begingroup$ This isn't true unless you are making a very narrow interpretation of the question. Please, then, explain what you understand the question to ask and be particularly careful of your quantifiers! $\endgroup$ – whuber Feb 15 '19 at 15:37
  • $\begingroup$ 1. @whuber’s solution goes much deeper than mine & seems to be in the spirit of the OP’s question. Bravo. 2. as @Henry points out, a notion of continuity cannot be ignored, to wit: 3. CasellaBerger tell us the right continuity is all we need for a CDF. 4. Which is implicit in @whuber’s discussion. 5, after the pyrotechnics of the non-affine transformations, Y=f(X) will still satisfy the requirement that a differential unit of x will be homogeneously distorted in creating a differential unit of y. So when all is said and done, the overall Jacobian remains constant everywhere. $\endgroup$ – Peter Leopold Feb 15 '19 at 23:54
  • $\begingroup$ It is not true the Jacobian must be everywhere constant. I think you misundertand my post, Peter: it shows only a few examples. There are many transformations $f:\mathbb{R}\to\mathbb{R}$ for which $f(X)$ is standard Normal when $X$ is, are almost everywhere infinitely differentiable, but have non-constant Jacobians. For instance, writing $\Phi$ for the standard Normal CDF and $F_{\chi^2(1)}$ for the chi-squared CDF (with one degree of freedom), $$f(x)=\Phi^{-1}\left(F_{\chi^2(1)}(x^2)\right)$$ is differentiable at all nonzero $x,$ has a varying derivative, and preserves standard Normality. $\endgroup$ – whuber Feb 16 '19 at 19:47
  • $\begingroup$ In pondering this some more, I suspect what you might be trying to claim is that when $f:\mathbb{R}\to\mathbb{R}$ is continuously differentiable and $f(X)$ has a Normal distribution for some particular random variable $X$ that is normally distributed, then $f$ must be affine. I believe that is true. The crux of the demonstration is to show $f$ cannot have any critical points. $\endgroup$ – whuber Feb 16 '19 at 22:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.