5
$\begingroup$

If I have a variable $X$ whose Gaussian distribution is known and let $f$ be a known function, is there a way to compute the distribution of $f(X)$ i.e. the resulting Gaussian distribution from this?

Is the result actually a Gaussian distribution?

$\endgroup$
  • 1
    $\begingroup$ $X\sim\mathcal N(\mu=0,~\sigma^2=1),~~f(X)=2$ $\endgroup$ – gung - Reinstate Monica May 9 '13 at 2:25
  • 1
    $\begingroup$ @gung Technically in your example $f(X)$ does have a Gaussian distribution. But the almost as simple function $X^2$ clearly is not Gaussian, because it can never be negative. Sree Prasanna, what exactly do you mean by "compute" $f(X)$? It can always be expressed as an integral--is that a "computation"? Or do you mean being able to give a closed-form formula whenever $f$ itself has a closed formula? Or do you mean just being able to compute probabilities numerically to any reasonable desired precision? Or something else? $\endgroup$ – whuber May 9 '13 at 3:31
  • $\begingroup$ @whuber Oh, my apologies, I didn't see you'd already done the same case in comments as I just did. I'll do another one as well, to make up for it. $\endgroup$ – Glen_b May 9 '13 at 7:04
  • 1
    $\begingroup$ @Dilip The "only when" is not correct. For instance, let $A$ be a Lebesgue measurable proper subset of positive reals and define $f(x)=-x$ whenever either $x \in A$ or $-x \in A$ and otherwise let $f(x)=x$. This $f$ does not have the form you propose, yet when $X$ has a Normal$(0,\sigma^2)$ distribution, so does $f(X)$. $\endgroup$ – whuber May 9 '13 at 14:31
  • $\begingroup$ @whuber Thanks for pointing out the erroneous statement. I remember the counterexample but it was not at the forefront of my mind. I am deleting my previous comment and replacing it with a corrected one. $\endgroup$ – Dilip Sarwate May 9 '13 at 17:58
5
$\begingroup$

I think you are talking about change of variables.

If X is a continuous r.v. with pdf $f_X(x)$ and sample space S, if $g: S \rightarrow T$ is an invertible transformation with differentiable inverse $h = g^{-1}$, and $Y = g(X)$, then Y is a continuous r.v. with pdf $f_Y(y)$ defined by

$f_Y(y) = f_X(h(y)) \cdot |h'(y)|$.

For example, suppose $X \sim exp(\lambda)$, what is the distribution of $X^2$?

Well, here we have $g: (0,\infty) \rightarrow (0,\infty)$ defined by $g(x) = x^2$. The inverse if $h(y) = g^{-1}(y) = \sqrt y$

Then $h'(y) = 0.5 y^{-1/2}$.

The density of X is $f_X(x) = \lambda e^{-\lambda x}$ so $f_X(h(y)) = \lambda e^{-\lambda \sqrt y}$.

And finally multiplying by the derivative gives $$ f_Y(y) = f_X(h(y)) \cdot |h'(y)| = \lambda e^{-\lambda \sqrt y} \cdot 0.5 y^{-1/2} $$

This simplifies to $$f_Y(y) = \frac{\lambda e^{-\lambda \sqrt y}}{2\sqrt y}$$

There is your density for a function of a random variable.

$\endgroup$
  • $\begingroup$ Yeah, that's the actual way to do it. I understood that after reading that in a book. Thanks. $\endgroup$ – SPRajagopal May 14 '13 at 9:28
  • $\begingroup$ @LindaH +1 Well constructed answer; it seems like perhaps you had more insight into the OP's underlying need than I did. $\endgroup$ – Glen_b May 14 '13 at 12:44
  • $\begingroup$ @Glen_b In that case, stats.stackexchange.com/questions/14483/… is extremely closely related. $\endgroup$ – whuber May 14 '13 at 17:47
5
$\begingroup$

Addressing the last question in particular --

1) Consider $X$ being standard Gaussian (mean 0, variance 1), and $U = \Phi(X)$ where $\Phi()$ is the standard normal cdf. Does $U$ have a Gaussian distribution? Let's simulate (in R in this case):

 u <- pnorm(rnorm(100000L))
 hist(u,n=300)

hist of Phi(Z)

... nope.

In fact you can work out that it must be standard uniform.


2) Consider $X$ being standard Gaussian (mean 0, variance 1), and $Y = f(X) = X^2$.

Does $Y$ have a Gaussian distribution? Let's simulate (in R):

 y <- rnorm(100000L)^2
 hist(y,n=300)

and what do we see?

histogram of squared standard normal

... nope.

In fact you can work out that it's chi-squared(1).


In particular circumstances, certain results imply you can obtain an approximately Gaussian distribution... but it's not the general case. For example, if the mean is many standard deviations from 0, (e.g. $X\sim N(5,0.1^2)$), then $Y=X^2$ is at least approximately normal:

 y <- rnorm(100000L,5,.1)^2
 hist(y,n=300)

square of normal, small CV

... and similarly $\exp(X)$, and $X^{1/3}$ and $log(X^2+\sqrt\pi)$, and a whole menagerie of other transformations will also give approximately Gaussian distributions in this case.

Edit:

You do get exact normality if you transform a Gaussian random variable with a linear transformation, so, as @DilipSarwate points out, if $X$ is Gaussian, $Y = a+bX$ is Gaussian (the case $b=0$ which was discussed by @gung and @whuber in comments is 'degenerate', but usually still counted as Gaussian when considering the whole location-scale family of Gaussians).

It's not quite the case that you can get to a normal only by linear transformation, though if you are restricted to monotonic transformations, I think this will be the case.

A nonlinear transformation of a normal that yields a normal:

Let $X$ be standard Gaussian. Let $F_1$ be the cdf of a $\chi^2_1$ random variable and $\Phi$ be the cdf of a standard normal. Then $\Phi^{-1}(F_1(X^2))$ is a nonlinear (and non-monotonic) transformation of $X$ where the resulting random variable is Gaussian.

y <- qnorm(pchisq(rnorm(100000L)^2,1))
hist(y,n=200)

nonlinear transform of normal giving normal


In respect of the first question, if $Y=h(X)$ you can in principle work out the distribution of $Y$. If $X$ is continuous and $h$ is invertible, it goes like this:

If $F_X$ is the cdf of $X$ then $P(Y\leq y) = P(h(X)\leq y) = P(X\leq h^{-1}(y)) = F_X(h^{-1}(y)))$.

From there one can work out the density by differentiation, leading to the standard result:

$$f_Y(y)= f_X(h^{-1}(y)) \left|\frac{d h^{-1}(y)}{dy}\right|$$

In other cases things are more complicated, but in some cases may still be doable.

$\endgroup$
  • 2
    $\begingroup$ Perhaps you should add a fourth case right after the approximate normality of $X^2$ pointing out that the distribution is exactly normal if and only if $f(X) = aX+b$. $\endgroup$ – Dilip Sarwate May 9 '13 at 11:53
  • $\begingroup$ Yes, indeed, it occurred to me that I should mention it shortly after I finished the previous edit ... but I couldn't come back to put it in until now. Thanks for the nudge. $\endgroup$ – Glen_b May 9 '13 at 17:51
  • 1
    $\begingroup$ You are welcome. However, please note that @whuber has just pointed out in a comment on the main question that "if and only if" is incorrect; it should be "if" only. $\endgroup$ – Dilip Sarwate May 9 '13 at 17:56
  • $\begingroup$ I hadn't seen those comments, but when I read your earlier comment on my answer it was immediately clear to me that a nonlinear transformation can still yield a normal; I have included an example in the edit to my answer that discusses the linear case (not the one @whuber mentioned though, I put in the simplest of several that occurred to me), though his example would have been a great one to include. I appreciate your comments. $\endgroup$ – Glen_b May 9 '13 at 18:10
  • $\begingroup$ I changed the placement of the word "only" in the sentence about nonlinear transformations possibly resulting in normal random variables. I think it makes the sentence read better. Please put back the way it was if you don't think so too. $\endgroup$ – Dilip Sarwate May 9 '13 at 18:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.