6
$\begingroup$

If the PDF of distribution of $x$ is $PDF_X$ and there is a mathematical relationship between $y$ and $x$:

$$y=f(x)$$

is it possible to find $PDF_Y$ ?

If yes, how to calculate it?

$\endgroup$
  • 2
    $\begingroup$ @wolfies what do you mean by closed-form solution? $\endgroup$ – ar2015 Dec 25 '15 at 6:21
  • 1
    $\begingroup$ No - is not possible. $\endgroup$ – wolfies Dec 25 '15 at 7:52
15
$\begingroup$

For a general function $h$, there is no direct formula to get the pdf of the random variable $Y=h(X)$ knowing the pdf of $X$. There is a formula in case when $h$ is a differentiable one-to-one mapping from the range (the support, I should say) of $X$ to the range of $Y$. I guess from your question that you don't know this formula, so let me give you the picture and the process to derive it.

Take for example a random variable $X \sim {\cal N}(\mu, \sigma^2)$ and set $Y=\exp(X)$. The animation below shows some simulations of $X$ and the corresponding values of $Y$. The density of $X$ is shown in blue and the one of $Y$ is shown in orange in the vertical direction.

enter image description here

Now the question is: knowing the density $f_{\textrm{blue}}$ of $X$, what is the density $f_{\textrm{orange}}$ of $Y$ ?

Taking a point $y$ in the range of $Y$, the density $f_{\textrm{orange}}$ provides the probability that $Y$ belongs to a small area $\mathrm{d}y$ around $y$ by the formula $$ \Pr(Y \in \mathrm{d}y) \approx f_{\textrm{orange}}(y)|\mathrm{d}y| $$ where $|\mathrm{d}y|$ denotes the length of the small interval $\mathrm{d}y$. This probability is the pink area on the figure below.

enter image description here

The probability $\Pr(Y \in \mathrm{d}y)$ also equals the probability $\Pr(X \in \mathrm{d}x)$, shown by the grey area below the blue curve, where $x=\log(y)$ because of $y=\exp(x)$, and $\mathrm{d}x$ is the small interval around $x$. This probability is given by $$ \Pr(X \in \mathrm{d}x) \approx f_{\textrm{blue}}(x)|\mathrm{d}x|. $$ It is clear that $|\mathrm{d}x| \neq |\mathrm{d}y|$. Remember that these two lengths are very small, hence the green function - let's call it $h$ instead of $\exp$ - is like a segment on the interval $\mathrm{d}x$, and the slope of this segment is the value $h'(x)$ of the derivative of $h$ at $x$. Therefore $|\mathrm{d}y| \approx h'(x)|\mathrm{d}x|$, and we finally get $$ \Pr(Y \in \mathrm{d}y) = \Pr(X \in \mathrm{d}x) \approx f_{\textrm{blue}}(x)\frac{|\mathrm{d}y|}{h'(x)}. $$ Expressing the right-hand side in terms of $y=h(x)$ instead of $x=h^{-1}(y)$, this gives $$ \Pr(Y \in \mathrm{d}y) \approx f_{\textrm{blue}}\bigl(h^{-1}(y)\bigr)\frac{|\mathrm{d}y|}{h'\bigl(h^{-1}(y)\bigr)}, $$ or, because of $\frac{1}{h'\bigl(h^{-1}(y)\bigr)}={(h^{-1})}'(y)$, this can be written $$ \Pr(Y \in \mathrm{d}y) \approx {(h^{-1})}'(y)\times f_{\textrm{blue}}\bigl(h^{-1}(y)\bigr)|\mathrm{d}y|. $$ By identifying this formula by the one defining the density of $Y$: $$ \Pr(Y \in \mathrm{d}y) \approx f_{\textrm{orange}}(y)|\mathrm{d}y|, $$ we finally get $$ \boxed{f_{\textrm{orange}}(y) = {(h^{-1})}'(y)\times f_{\textrm{blue}}\bigl(h^{-1}(y)\bigr)}. $$ This is the so-called change of variables formula.

Be careful about one point: this formula is not correct in general. In my example, the factor $k$ relating $|\mathrm{d}x|$ and $|\mathrm{d}y|$ by the approximative equality $|\mathrm{d}y| \approx k|\mathrm{d}x|$ is $k = h'(x)$ because $h'(x)>0$ in this example ($h$ is increasing), and one has to take $-h'(x)$ if $h'(x) <0$. The general formula includes the absolute value: $$ \boxed{f_{\textrm{orange}}(y) = \bigl|{(h^{-1})}'(y)\bigr|\times f_{\textrm{blue}}\bigl(h^{-1}(y)\bigr)}. $$

$\endgroup$
  • $\begingroup$ I'm not sure that $f$ has to be a one-to-one mapping. If it has more than one inverse, you can sum over the inverses. E.g. see my answer $\endgroup$ – Jorge Leitao Dec 25 '15 at 20:43
  • $\begingroup$ @J.C.Leitão I just mean I give the formula for the case of a 1-1 mapping. Otherwise I can't say $\Pr(Y \in dy)=\Pr(X \in dx)$. $\endgroup$ – Stéphane Laurent Dec 25 '15 at 21:18
2
$\begingroup$

Suppose $X$ has a standard normal distribution, then pdf of $X$ is $f(x)=\frac{1}{\sqrt{2\pi}}e^{-\frac{1}{2}x^2}$

Suppose the mathematical relationship between the random variable $Y$ and $X$ is $Y=2X$. It will be easy to find the pdf of $Y$.

Basically there are two methods to find the pdf of Y

1. use the CDF then take derivative the CDF.

2. Use variable transformation directly (use Jacobian)

I will show the second method here.

$y=2x \Rightarrow x=\frac{1}{2}y$

$J=\frac{dx}{dy}=\frac{1}{2}$

$f(y)=\frac{1}{\sqrt{2\pi}}e^{-\frac{1}{2}(\frac{y}{2})^2}|J|=\frac{1}{2\sqrt{2\pi}}e^{-\frac{1}{8}y^2}$

You can see the pdf of $Y$ is $\frac{1}{2\sqrt{2\pi}}e^{-\frac{1}{8}y^2}$

$\endgroup$
2
$\begingroup$

Yes, it is possible. Let's see how.

When $Y = f(X)$, the conditional P(Y|X) can be written as

$$P(Y=y|x) = \delta(y - f(x))$$

where $\delta$ is the Dirac delta.

The PDF of Y can be obtained by marginalising over X:

$$ P(Y=y) = \int P(y|x) P(x)dx = \int \delta(y - f(x))P(x)dx $$

To compute such integral, we can use that

$$\int h(x)\delta(g(x)) dx = \sum_i \frac{h(x_i)}{|g'(x_i)|}$$

where $x_i$ are the roots of $g$. Replacing it in our marginal with $g(x) = y - f(x)$, and $h(x)=P(x)$, we get

$$ P(Y=y) = \sum_{i=1}^N \frac{P(x_i)}{|f'(x_i)|} $$

where $x_i$ are the solutions of $y = f(x)$.

This is as far as we can go. Nevertheless, this formula evidences that:

  • the problem can be treated as any other problem of probability: you define the random variables, the conditional probabilities, and compute the marginal;
  • the solution does not necessarily have a closed formula, but it does not require $f$ to be bijective;
  • when the function is bijective (single root), this solution reduces to the solution @Stéphane Laurent gave.

I find this solution nice because I don't have to remember it; it is a consequence of the definition of $Y$ and $P(Y|X)$.

Let's take an example: X is uniformly in $x \in [-1/2, 1/2]$, $Y = f(X) = X^2 \in [0, 1/4]$. Compute $P(Y=y)$.

There are two solutions of $y=x^2$ in the interval $x \in [-1/2, 1/2]$: $\{-\sqrt{y}, \sqrt{y}\}$. Moreover, the derivative of $f(X)$ is $2x$. Using the equation above, we get

$$P(y) = \sum_{i=1}^{2} \frac{P\left(x_{i}\right)}{|f'(x_{i})|} = \left( \frac{1}{2\sqrt{y}} + \frac{1}{2\sqrt{y}} \right) = \frac{1}{\sqrt{y}}$$

which can be confirmed e.g. in mathematica:

Show[{Histogram[
   RandomVariate[UniformDistribution[{-1/2, 1/2}], 100000]^2, 
   Automatic, "PDF"], 
  Plot[1/Sqrt[y], {y, 0.001, 1/4}, PlotStyle -> {Red, Thick}, 
   PlotRange -> All]}]

enter image description here

If X is uniform in $[0, 1]$ instead, there is only one solution and we get

$$P(y) = \frac{1}{2\sqrt{y}}$$

$\endgroup$
  • $\begingroup$ $\Pr(Y=y)=0$ if $Y$ has a pdf. $\endgroup$ – Stéphane Laurent Dec 25 '15 at 21:18
2
$\begingroup$

As I understand your question, you ask about situation where we have random variable $X$, its density function $f(x)$, and you are interested of finding probability density function of random variable $Y$ defined as $Y = f(X)$.

Such relationship is pretty straightforward for cumulative distribution function, but does not have to exist for probability density function. Recall that

function is a relation between a set of inputs and a set of permissible outputs with the property that each input is related to exactly one output. (Wikipedia, italics added)

and look at the two figures below showing PDF of standard normal $f(x)$. Every $x$-axis value is related to exactly one $y$-axis value (left figure), but some $y$-axis values are related to more than one $x$-axis value (right figure), so the inverse relationship is not a function. This does not have to be true for all the functions, but only to the ones that are not one-to-one mappings.

enter image description here

If you are interested in probability of observing $f(x)$ values, it can be obtained by simulation. To compute probabilities sample values from random variable $X$ and pass them through density function $f(\cdot)$ the same way as you could make any other transformations of $X$. This enables you to obtain $y=f(x)$ and then compute empirical probabilities for $y$ values. Example of R code for normal distribution is provided below.

hist(dnorm(rnorm(1e5)))

enter image description here

$\endgroup$
  • 1
    $\begingroup$ I think your choice of $f$ is possibly confusing, because it is a pdf. $\endgroup$ – Stéphane Laurent Dec 25 '15 at 14:18
  • $\begingroup$ @StéphaneLaurent thanks, I made the wording more clear. $\endgroup$ – Tim Dec 25 '15 at 21:22
  • $\begingroup$ Tim, I still don't understand. There are two pdf's in the OP, and a transformation $f$. But you take for $f$ a pdf, that's why I said it is confusing. It would be clearer for example if you took $f(x)=x^2$. $\endgroup$ – Stéphane Laurent Dec 26 '15 at 12:27
  • $\begingroup$ I may be wrong, but I'm under the impression you misread the question, and you are answering to the question of the law of $f(X)$ when $f$ is the density of $X$. $\endgroup$ – Stéphane Laurent Dec 26 '15 at 13:40
  • $\begingroup$ @StéphaneLaurent as I understood the question $f$ is is a density function for $X$. Unfortunately OP did not make it clear in his/her question and did not provide any follow-up comments to my answer. $\endgroup$ – Tim Dec 26 '15 at 15:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.