Transformations that preserve normality for all multivariate normal distributions

Question

I am looking for measurable invertible functions $f:\mathbb{R}^n \rightarrow \mathbb{R}^n$ such that for all (non-degenerate) n-dimensional Gaussian random variables $X$, we have that $f(X)$ is still a Gaussian random variable. I conjecture that $f$ has to be affine. Is this true or are additional assumptions on $f$ needed?

Answer 1

The paper, cited by Firebug, answers affirmatively that the functions that preserve multivariate normality are linear in the sense that, if $\mathcal M:= \{P_{\boldsymbol\theta}\mid \boldsymbol\theta\in\Theta\wedge P_{\boldsymbol\theta}\equiv \mathtt{MultNorm}(\boldsymbol\theta)\}$ where $\Theta:= \{(\boldsymbol\mu,\boldsymbol\Sigma)\in \mathrm{l\!R}^p\times \mathscr P_p\} ,~ \mathscr P_p $ being the collection of all positive definite matrices of order $p,$ and $g:\mathrm{l\!R}^p\to \mathrm{l\!R}^p $ is a bijective bimeasurable transformation mapping $\mathbf X$ whose law $\mathcal L(\mathbf X)\in \mathcal M$ such that $\mathcal L(g(\mathbf X))\in\mathcal M.$

However, if the question is for which set $A\times B \subseteq \mathrm{l\!R}^p\times \mathscr S_p ,$ one could conclude $g$ is linear that preserves multinormality, the paper does provide a stronger result.

This post would look into the approach taken to prove the result and provide an outline of the same.

WLOG, the following lemmas would be shown for $\boldsymbol\mu=\boldsymbol 0. $

Lemma $1:$ For a fixed $\boldsymbol\Sigma_1, \boldsymbol\Sigma_2\in \mathscr P_p,$ if $\mathbf X\sim \texttt{MutltNorm}(\mathbf 0, \boldsymbol \Sigma_i)$ and $g(\mathbf X)\sim\texttt{MutltNorm}(\boldsymbol\eta_i, \boldsymbol \Psi_i) \in \mathcal M, ~i= \overline{1,2},$ then almost surely $$\mathbf x^\top\left(\boldsymbol \Sigma_1^{-1}-\boldsymbol \Sigma_2^{-1} \right)\mathbf x = (g(\mathbf x) - \boldsymbol\eta_1)^\top\boldsymbol\Psi_1^{-1}(g(\mathbf x) - \boldsymbol\eta_1)- (g(\mathbf x) - \boldsymbol\eta_2)^\top\boldsymbol\Psi_2^{-1}(g(\mathbf x) - \boldsymbol\eta_2) + \ln\vert \boldsymbol\Psi_1\boldsymbol\Sigma_2\vert/\vert \boldsymbol\Sigma_1\boldsymbol\Psi_2\vert.$$

Note $\lambda\circ g\ll \lambda$ and $\lambda(\mathrm d\mathbf y) = \lambda(g(\mathrm d\mathbf x)) $ as $g$ is bijective and bimeasurable; let the corresponding RN derivative be denoted by $k(\mathbf x).$ Therefore $\varphi(\mathbf x; \mathbf 0, \boldsymbol\Sigma_i) =\varphi(g(\mathbf x); \mathbf \eta_i, \boldsymbol\Psi_i )\cdot k(\mathbf x); $ divide this relation corresponding to $i = 1$ and $2$ to get the result.

$\blacksquare$

Lemma $2:$ If $\mathbf G:=\boldsymbol \Sigma_1^{-1}-\boldsymbol \Sigma_2^{-1} $ and $\mathbf H:= \boldsymbol \Psi_1^{-1}-\boldsymbol \Psi_2^{-1} ,$ then

• $\bf G\cong H;$

• $\boldsymbol\eta_1 = \boldsymbol\eta_2,$ that is, $\boldsymbol\eta$ is independent of $\boldsymbol\Sigma;$

• Almost surely $$\mathbf x^\top\mathbf G\mathbf x = (g(\mathbf x) - \boldsymbol\eta_1)^\top\mathbf H(g(\mathbf x) - \boldsymbol\eta_1).$$

By Lemma $1.,$ it is true almost surely that $\mathbf x^\top\mathbf G\mathbf x =(g(\mathbf x) - \boldsymbol\eta_1)^\top\mathbf H(g(\mathbf x) - \boldsymbol\eta_1) - 2(\boldsymbol\eta_1-\boldsymbol\eta_2)^\top\boldsymbol\Psi_{2}^{-1}(g(\mathbf x) - \boldsymbol\eta_1) + c_2$ where $c_i = (-1)^{i+1}(\boldsymbol\eta_1-\boldsymbol\eta_2)^\top\boldsymbol\Psi_{2}^{-1}(\boldsymbol\eta_1-\boldsymbol\eta_2) +\ln \vert \boldsymbol\Psi_1\boldsymbol\Sigma_2\vert/\vert \boldsymbol\Sigma_1\boldsymbol\Psi_2\vert.$ The right hand side is of the form $\mathbf x^\top \mathbf A\mathbf x +\mathbf a^\top\mathbf X + d,$ where $\bf X$ follows nondegenerate multivariate normal. Equating the mgf of both sides, the latter of which is derived in $[\rm II]$ yields $$\left\vert\mathbf I -2t\boldsymbol{\Sigma}_1^{1/2}\boldsymbol{\mathbf G\Sigma}_1^{1/2}\right\vert^{-1/2}=e^{\chi(t)}\left\vert\mathbf I -2t\boldsymbol{\Psi}_1^{1/2}\boldsymbol{\mathbf H\Psi}_1^{1/2}\right\vert^{-1/2},$$ where $ \chi(t) := 2t^2(\boldsymbol\eta_1-\boldsymbol\eta_2)^\top \boldsymbol\Psi_2^{-1}\left(\boldsymbol\Psi_1^{-1}-2t\mathbf H\right)^{-1}\boldsymbol\Psi_2^{-1}(\boldsymbol\eta_1-\boldsymbol\eta_2) + c_2t. $ For values of $t$ sufficiently close to $t= 0, \mathbf I -2t\boldsymbol{\Sigma}_1^{1/2}\boldsymbol{G\Sigma}_1^{1/2}$ is non-singular and thus in a suitable nbd of $t= 0,$ $$\left\vert\mathbf I -2t\boldsymbol{\Psi}_1^{1/2}\boldsymbol{\mathbf H\Psi}_1^{1/2}\right\vert/\left\vert\mathbf I -2t\boldsymbol{\Sigma}_1^{1/2}\boldsymbol{\mathbf G\Sigma}_1^{1/2}\right\vert=\exp(2(\chi(t)).$$ As explained in this Math.SE post, this leads to $\chi(t)\equiv 0.$ This means $\left\vert\mathbf I -2t\boldsymbol{\Psi}_1^{1/2}\boldsymbol{\mathbf H\Psi}_1^{1/2}\right\vert= \left\vert\mathbf I -2t\boldsymbol{\Sigma}_1^{1/2}\boldsymbol{\mathbf G\Sigma}_1^{1/2}\right\vert$ for all $t.$ Therefore the eigenvalues of $ \boldsymbol{\Psi}_1^{1/2}\boldsymbol{\mathbf H\Psi}_1^{1/2}$ are equal to that of $\boldsymbol{\Sigma}_1^{1/2}\boldsymbol{\mathbf G\Sigma}_1^{1/2};$ by Sylvester's law of inertia, $\bf G\cong H.$ As $\chi(t)\equiv 0,$ this implies $c_2=\chi^\prime(0) = 0.$ Similarly, $c_1 = 0.$ From $0= c_1- c_2 =(\boldsymbol\eta_1-\boldsymbol\eta_2)^\top\left(\boldsymbol\Psi_{1}^{-1} + \boldsymbol\Psi_{2}^{-1}\right)(\boldsymbol\eta_1-\boldsymbol\eta_2) $ implies the last two to be proved.

$\blacksquare$

The last implication results this:

Lemma $3:$ For each $\mathbf x\in \mathrm{l\!R}^p,$ there exist unique symmetric matrices $\mathbf B_{ij}$ for $ 1\leq i\leq j\leq p$ such that $$x_ix_j = (g(\mathbf x) - \boldsymbol\eta)^\top\mathbf B_{ij}(g(\mathbf x) - \boldsymbol\eta).$$ Furthermore, $\operatorname{rank}(\mathbf{B}_{ii} )= 1.$

Coming to the main results, first a single $\boldsymbol\mu\in \mathrm{l\!R}^p$ is fixed. What would $g$ that preserves normality for the corresponding distributions look like?

Theorem $1:$ For a fixed $\boldsymbol\mu,$ a preserving bijective bimeasurable transformation $g$ would be of the form almost surely $g(\mathbf x) = h(\mathbf x-\boldsymbol\mu)\mathbf A(\mathbf x-\boldsymbol\mu)+\boldsymbol\eta,$ where $\mathbf A\in\mathsf{GL}(p), \boldsymbol\eta\in \mathrm{l\!R}^p$ and $h:\mathrm{l\!R}^p\to \mathrm{l\!R}$ such that $h(-\mathbf x)= h(\mathbf x)$ and $\operatorname{range}(h)\equiv\{1,-1\} .$

Note first that $\mathbf B_{ji} = \mathbf B_{ij}^\top = \mathbf B_{ij}.$ By lemma $3.,$ there exists a unique $\mathbf b_i\in \mathrm{l\!R}^p $ such that $\mathbf B_{ii} = \mathbf b_i\mathbf b_i^\top$ except for sign change. And $x_i= \pm \mathbf b_i^\top(g(\mathbf x)- \boldsymbol\eta)$ almost surely. $\mathbf B_{1i} = \pm \frac12\left(\mathbf b_1\mathbf b_i^\top +\mathbf b_i\mathbf b_1^\top\right).$ This uniquely determines the signs of $\mathbf b_i$ once the sign of $\mathbf b_1$ is known. Thus with $\mathbf C:= (\mathbf b_1,\ldots, \mathbf b_p)^\top$ being invertible, almost surely $\mathbf x = h(\mathbf x)(g(\mathbf x)-\boldsymbol \eta),$ with $\operatorname{range}{h}\equiv\{+1,-1\}. $ Bijectiveness of $g$ implies $g(\mathbf x)\ne g(-\mathbf x)$ implying $h(\mathbf x)=h(-\mathbf x).$ Thus with $\mathbf A:=\mathbf C^{-1} , ~g(\mathbf x)= h(\mathbf x)\mathbf A\mathbf x+\boldsymbol\eta.$

$\blacksquare$

The next result shows that fixing two distinct $\boldsymbol \mu$s would turn $g$ linear almost surely.

Theorem $2:$ For fixed $\boldsymbol\mu_1, \boldsymbol\mu_2\in \mathrm{l\!R}^p, $ if $g$ is a preserving bijective bimeasurable transformation for the subfamily of $\mathcal M$ determined by $\{(\boldsymbol\mu,\boldsymbol\Sigma)\in \{\boldsymbol\mu_1, \boldsymbol\mu_2\}\times \mathscr P_p\},$ then almost surely $g(\mathbf x) = \mathbf A\mathbf x+\mathbf a,$ where $\mathbf A\in\mathsf{GL}(p), \mathbf a\in\mathrm{l\!R}^p.$

If Theorem $1.,$ is applied for each $\boldsymbol\mu_i, ~i\in\overline{1,2},$ then there exists $\mathbf A_i\in\mathsf{GL}(p)$ and $\boldsymbol\eta_i\in\mathrm{l\!R}^p$ such that almost surely $g(\mathbf x)=h_i(\mathbf x-\boldsymbol \mu_i)\mathbf A_i(\mathbf x-\boldsymbol \mu_i)+\boldsymbol\eta_i.$ For $k, l\in\overline{1,2}, $ consider the four disjoint sets $F_{kl}\in\{\mathbf x\in\mathrm{l\!R}^p\mid h_1(\mathbf x-\boldsymbol \mu_1) = (-1)^k, ~h_2(\mathbf x-\boldsymbol \mu_2) = (-1)^l\}.$ Since $\sqcup F_{kl} = \mathrm{l\!R}^p,$ at least one $\lambda(F_{kl})>0. $ Assume that set is $F_{00};$ then almost surely, by definition of the set, $\mathbf A_1(\mathbf x-\boldsymbol \mu_1)+\boldsymbol\eta_1= \mathbf A_2(\mathbf x-\boldsymbol \mu_2)+\boldsymbol\eta_2.$

Choose $\mathbf x^{\{s\}}\in F_{00}, ~s\in\overline{0,1, \ldots, p}$ such that they don't lie in a hyperplane. Also, wlog, assume $\mathbf x^{\{s\}}~s\in\overline{1, \ldots, p}$ are linearly independent; so $\mathbf x^{\{0\}} = \sum_{j=1}^pc_j\mathbf x^{\{j\}}$ and since these aren't in a hyperplane, $\sum_j c_j \ne 1.$ For each $s, ~(\mathbf A_1-\mathbf A_2)\mathbf x^{\{s\}} = \mathbf A_1\boldsymbol\mu_1-\mathbf A_2\boldsymbol\mu_2 +\boldsymbol\eta_2-\boldsymbol\eta_1.$ Multiplying $c_s$ to both sides of this relation and summing them up would yield $(\mathbf A_1-\mathbf A_2)\mathbf x^{\{0\}} = \left(\sum_j c_j\right)(\mathbf A_1\boldsymbol\mu_1-\mathbf A_2\boldsymbol\mu_2 +\boldsymbol\eta_2-\boldsymbol\eta_1),$ prompting the sum in the second parenthesis to be $\bf 0,$ which further implies $\mathbf A_1=\mathbf A_2$ owing to $\mathbf x^{\{s\}}, s\in\overline{1,\ldots, p},$ being LIs.

The uniqueness of the other components of $g$ follows from showing that all other $F_{kl}, ~k,l\ne 0$ are of zero measures.

$\blacksquare$

Corollary $1:$ If $g$ preserves $\mathcal M$ for all $\Theta,$ then $g(\mathbf x)= \mathbf A\mathbf x+\mathbf a,$ where $\mathbf A\in\mathsf{GL}(p), \mathbf a\in\mathrm{l\!R}^p.$

--

References:

$\rm[I]$ Transformations preserving normality and Wishart-ness, Seiji Nabeya, Takeaki Kariya, JOURNAL OF MULTIVARIATE ANALYSIS $20, ~251-264 ~(1986),$ [doi].

$\rm[II] $ Distribution of Quadratic Forms, Marc S Paolella, Wiley Online Library, $2018,$ [doi].