Distribution of Y from distribution of X

Question

If the PDF of distribution of $x$ is $PDF_X$ and there is a mathematical relationship between $y$ and $x$:

$$y=f(x)$$

is it possible to find $PDF_Y$ ?

If yes, how to calculate it?

Answer 1

For a general function $h$, there is no direct formula to get the pdf of the random variable $Y=h(X)$ knowing the pdf of $X$. There is a formula in case when $h$ is a differentiable one-to-one mapping from the range (the support, I should say) of $X$ to the range of $Y$. I guess from your question that you don't know this formula, so let me give you the picture and the process to derive it.

Take for example a random variable $X \sim {\cal N}(\mu, \sigma^2)$ and set $Y=\exp(X)$. The animation below shows some simulations of $X$ and the corresponding values of $Y$. The density of $X$ is shown in blue and the one of $Y$ is shown in orange in the vertical direction.

Now the question is: knowing the density $f_{\textrm{blue}}$ of $X$, what is the density $f_{\textrm{orange}}$ of $Y$ ?

Taking a point $y$ in the range of $Y$, the density $f_{\textrm{orange}}$ provides the probability that $Y$ belongs to a small area $\mathrm{d}y$ around $y$ by the formula $$ \Pr(Y \in \mathrm{d}y) \approx f_{\textrm{orange}}(y)|\mathrm{d}y| $$ where $|\mathrm{d}y|$ denotes the length of the small interval $\mathrm{d}y$. This probability is the pink area on the figure below.

The probability $\Pr(Y \in \mathrm{d}y)$ also equals the probability $\Pr(X \in \mathrm{d}x)$, shown by the grey area below the blue curve, where $x=\log(y)$ because of $y=\exp(x)$, and $\mathrm{d}x$ is the small interval around $x$. This probability is given by $$ \Pr(X \in \mathrm{d}x) \approx f_{\textrm{blue}}(x)|\mathrm{d}x|. $$ It is clear that $|\mathrm{d}x| \neq |\mathrm{d}y|$. Remember that these two lengths are very small, hence the green function - let's call it $h$ instead of $\exp$ - is like a segment on the interval $\mathrm{d}x$, and the slope of this segment is the value $h'(x)$ of the derivative of $h$ at $x$. Therefore $|\mathrm{d}y| \approx h'(x)|\mathrm{d}x|$, and we finally get $$ \Pr(Y \in \mathrm{d}y) = \Pr(X \in \mathrm{d}x) \approx f_{\textrm{blue}}(x)\frac{|\mathrm{d}y|}{h'(x)}. $$ Expressing the right-hand side in terms of $y=h(x)$ instead of $x=h^{-1}(y)$, this gives $$ \Pr(Y \in \mathrm{d}y) \approx f_{\textrm{blue}}\bigl(h^{-1}(y)\bigr)\frac{|\mathrm{d}y|}{h'\bigl(h^{-1}(y)\bigr)}, $$ or, because of $\frac{1}{h'\bigl(h^{-1}(y)\bigr)}={(h^{-1})}'(y)$, this can be written $$ \Pr(Y \in \mathrm{d}y) \approx {(h^{-1})}'(y)\times f_{\textrm{blue}}\bigl(h^{-1}(y)\bigr)|\mathrm{d}y|. $$ By identifying this formula by the one defining the density of $Y$: $$ \Pr(Y \in \mathrm{d}y) \approx f_{\textrm{orange}}(y)|\mathrm{d}y|, $$ we finally get $$ \boxed{f_{\textrm{orange}}(y) = {(h^{-1})}'(y)\times f_{\textrm{blue}}\bigl(h^{-1}(y)\bigr)}. $$ This is the so-called change of variables formula.

Be careful about one point: this formula is not correct in general. In my example, the factor $k$ relating $|\mathrm{d}x|$ and $|\mathrm{d}y|$ by the approximative equality $|\mathrm{d}y| \approx k|\mathrm{d}x|$ is $k = h'(x)$ because $h'(x)>0$ in this example ($h$ is increasing), and one has to take $-h'(x)$ if $h'(x) <0$. The general formula includes the absolute value: $$ \boxed{f_{\textrm{orange}}(y) = \bigl|{(h^{-1})}'(y)\bigr|\times f_{\textrm{blue}}\bigl(h^{-1}(y)\bigr)}. $$

Answer 2

Suppose $X$ has a standard normal distribution, then pdf of $X$ is $f(x)=\frac{1}{\sqrt{2\pi}}e^{-\frac{1}{2}x^2}$

Suppose the mathematical relationship between the random variable $Y$ and $X$ is $Y=2X$. It will be easy to find the pdf of $Y$.

Basically there are two methods to find the pdf of Y

1. use the CDF then take derivative the CDF.

2. Use variable transformation directly (use Jacobian)

I will show the second method here.

$y=2x \Rightarrow x=\frac{1}{2}y$

$J=\frac{dx}{dy}=\frac{1}{2}$

$f(y)=\frac{1}{\sqrt{2\pi}}e^{-\frac{1}{2}(\frac{y}{2})^2}|J|=\frac{1}{2\sqrt{2\pi}}e^{-\frac{1}{8}y^2}$

You can see the pdf of $Y$ is $\frac{1}{2\sqrt{2\pi}}e^{-\frac{1}{8}y^2}$

Answer 3

Yes, it is possible. Let's see how.

When $Y = f(X)$, the conditional P(Y|X) can be written as

$$P(Y=y|x) = \delta(y - f(x))$$

where $\delta$ is the Dirac delta.

The PDF of Y can be obtained by marginalising over X:

$$ P(Y=y) = \int P(y|x) P(x)dx = \int \delta(y - f(x))P(x)dx $$

To compute such integral, we can use that

$$\int h(x)\delta(g(x)) dx = \sum_i \frac{h(x_i)}{|g'(x_i)|}$$

where $x_i$ are the roots of $g$. Replacing it in our marginal with $g(x) = y - f(x)$, and $h(x)=P(x)$, we get

$$ P(Y=y) = \sum_{i=1}^N \frac{P(x_i)}{|f'(x_i)|} $$

where $x_i$ are the solutions of $y = f(x)$.

This is as far as we can go. Nevertheless, this formula evidences that:

• the problem can be treated as any other problem of probability: you define the random variables, the conditional probabilities, and compute the marginal;
• the solution does not necessarily have a closed formula, but it does not require $f$ to be bijective;
• when the function is bijective (single root), this solution reduces to the solution @Stéphane Laurent gave.

I find this solution nice because I don't have to remember it; it is a consequence of the definition of $Y$ and $P(Y|X)$.

Let's take an example: X is uniformly in $x \in [-1/2, 1/2]$, $Y = f(X) = X^2 \in [0, 1/4]$. Compute $P(Y=y)$.

There are two solutions of $y=x^2$ in the interval $x \in [-1/2, 1/2]$: $\{-\sqrt{y}, \sqrt{y}\}$. Moreover, the derivative of $f(X)$ is $2x$. Using the equation above, we get

$$P(y) = \sum_{i=1}^{2} \frac{P\left(x_{i}\right)}{|f'(x_{i})|} = \left( \frac{1}{2\sqrt{y}} + \frac{1}{2\sqrt{y}} \right) = \frac{1}{\sqrt{y}}$$

which can be confirmed e.g. in mathematica:

Show[{Histogram[
   RandomVariate[UniformDistribution[{-1/2, 1/2}], 100000]^2, 
   Automatic, "PDF"], 
  Plot[1/Sqrt[y], {y, 0.001, 1/4}, PlotStyle -> {Red, Thick}, 
   PlotRange -> All]}]

If X is uniform in $[0, 1]$ instead, there is only one solution and we get

$$P(y) = \frac{1}{2\sqrt{y}}$$

Answer 4

As I understand your question, you ask about situation where we have random variable $X$, its density function $f(x)$, and you are interested of finding probability density function of random variable $Y$ defined as $Y = f(X)$.

Such relationship is pretty straightforward for cumulative distribution function, but does not have to exist for probability density function. Recall that

function is a relation between a set of inputs and a set of permissible outputs with the property that each input is related to exactly one output. (Wikipedia, italics added)

and look at the two figures below showing PDF of standard normal $f(x)$. Every $x$-axis value is related to exactly one $y$-axis value (left figure), but some $y$-axis values are related to more than one $x$-axis value (right figure), so the inverse relationship is not a function. This does not have to be true for all the functions, but only to the ones that are not one-to-one mappings.

If you are interested in probability of observing $f(x)$ values, it can be obtained by simulation. To compute probabilities sample values from random variable $X$ and pass them through density function $f(\cdot)$ the same way as you could make any other transformations of $X$. This enables you to obtain $y=f(x)$ and then compute empirical probabilities for $y$ values. Example of R code for normal distribution is provided below.

hist(dnorm(rnorm(1e5)))