1
$\begingroup$

We cannot work with multivariate normal distribution unless all variates follow normal distribution. Can we somehow transform all to normal?

$\endgroup$
  • 1
    $\begingroup$ To get a multivariate normal you need all linear combinations of the marginal variables to be normal and not just the marginal variables themselves. To make a continuous univariate random variable normal just take the inverse cumulative distribution to get a uniform variable on [0, 1] and then apply the cumulative normal to it. $\endgroup$ – Michael Chernick Jul 28 '17 at 1:09
  • $\begingroup$ If you know the marginal distributions that transformation Michael suggests would get you normal margins but it won't give you the desired multivariate normality -- the result would only be multivariate normal if you started with a multivariate distribution which had a Gaussian copula to begin with. He's completely correct in what he's saying, but I wanted to emphasize that he's saying that the transformation in his second sentence doesn't give you the desired result. [You'd also require some way of transforming between copulas to achieve it in general.] $\endgroup$ – Glen_b Jul 28 '17 at 1:13
5
$\begingroup$

I very much like the way you are thinking, but the answer is no. Given a sample space $E$. Any probability distribution $X$ over the sample space $E$ with probability measure $\mu_x$ can be transformed into any other probability distribution $Y$ over the sample space $E$ with probability measure $\mu_y$ as long as the probability measures are absolutely continuous.

Example 1: You cannot transform a discrete distribution (like the Poisson distribution) into a normal distribution because the Poisson distribution only has non-zero probability on the natural numbers whereas a 1 dimensional normal distribution has non-zero probability over the entire real line.

In addition, under certain circumstances, a distribution $X$ over sample space $E$ with probability measure $\mu_x$ can be transformed into a distribution $Y$ over sample space $F$ and probability measure $\mu_y$ so long as there is a topological equivalence between the two spaces $E$ and $F$ such that the probability measures $\mu_x$ and $\mu_y$ can be made absolutely continuous. For example, the positive real line $(0, \infty)$ (note I am excluding zero) can be transformed to the entire real line $(-\infty, \infty)$ using the log transform which is a continuous bijective mapping between the two spaces.

Example 2: Think about how the log-normal distribution can be transformed to the normal distribution by taking the log-transform of a log-normal random variable. Note however that the positive real line including zero $[0, \infty)$ is not topologically equivalent to the entire real line $(-\infty, \infty)$ due to the closed boundary at zero and as such something like a truncated normal defined over $[0, \infty)$ cannot be transformed to a normal distribution over $(-\infty, \infty)$.

To summarize this: The two distribution need to have zero and non-zero probability in the same places or there needs to be a continuous bijective mapping between the sample spaces such that the two distribution can be made to have zero and non-zero probability in the same places.

For an intuitive background on the measure theory needed to understand this, check out a recent blog post: Measure Theory Made Ridiculously Simple. In particular, at the end of the post I show how a transformation of variables is actually a change of measure; however, there to simplify my treatment I did not discuss the requirement of topological equivalency between sample spaces.

Update Based on Comments: It has been brought to my attention that the question may be referring to the existence of transformations that operate on each variate separately. In such cases in addition to the existence of a continuous bijective mapping between two sample spaces $E$ and $F$ (where $F = R^n$ as the poster is discussing multivariate normality) there is an added requirement on the mapping. For notational purposes lets assume the question refers to a starting multivariate distribution $X=(x_1, \dots, x_n)$ over a sample space $E^n = E_1\times\dots\times E_n$.

We now have the following restrictions on the transformation (mapping):

  1. As before we require that there exists a bijective continuous mapping between $E$ and $R^n$ (This can trivially be the identity map if $E = R^n$).
  2. As before we require that this mapping is such that $\mu_x$ (the probability measure of the starting distribution) is absolutely continuous with a transformed measure $\mu_y$ (the gaussian measure over $R^n$.
  3. We now also require that the mapping has the form $f(x_1, \dots, x_n)=f_1(x_1)\circ\dots\circ f_n(x_n)$ such that $f:E^n \rightarrow F^n$ and where $f_i : E^i \rightarrow F^i$ such that the transform is essentially operating on each variate separately.

Adding this further restriction simply solidifies the conclusion that such a transformation may not always be possible.

$\endgroup$
  • 1
    $\begingroup$ Your characterization "have zero and non-zero probability in the same places" could easily be misunderstood. As a simple example, a discrete distribution supported on $\{0,1\}$ is readily transformed to a discrete distribution supported on $\{-1,1\}$ but evidently these distributions are singular with respect to each other. What, then, do you really mean to say? $\endgroup$ – whuber Jul 28 '17 at 13:59
  • $\begingroup$ Thank you for that point. You are correct, I needed to discuss the possibility of the transformation of the underlying sample space. I hope my updated answer is more clear. $\endgroup$ – jds Jul 28 '17 at 16:33
  • 1
    $\begingroup$ That clears it up pretty well. However, I don't think you have yet addressed the actual question. Because it refers to "variates," the clear intention is that any transformation will be applied separately to each variable--for otherwise the distinctions between the variables disappear. With more than two variables, it is not possible in this limited way to transform any absolutely continuous distribution into any other. That is the point both Michael Chernick and @Glen_b are making in their comments to the question. $\endgroup$ – whuber Jul 28 '17 at 16:40
  • $\begingroup$ I am not sure what you mean when you say "otherwise the distinctions between the variables disappear". Lets assume the question refers to a starting multivariate distribution $X=(x_1, \dots, x_n)$ over a sample space $E^n = E_1\times\dots\times E_n$. If you want to restrict to only transformations of the form $f(x)=f_1(x_1)\circ\dots\circ f_n(x_n)$ such that $f:E^n \rightarrow F^n$ and where $f_i : E^i \rightarrow F^i$ such that the transform is essentially operating on each variate separately, you totally can, it just a further restriction on my above answer. -Happy to update, just unclear $\endgroup$ – jds Jul 28 '17 at 16:50
  • $\begingroup$ The point is that this "further restriction" makes your answer incorrect. $\endgroup$ – whuber Jul 28 '17 at 16:52

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.