# Derivation of change of variables of a probability density function?

In the book pattern recognition and machine learning (formula 1.27), it gives

$$p_y(y)=p_x(x) \left | \frac{d x}{d y} \right |=p_x(g(y)) | g'(y) |$$ where $x=g(y)$, $p_x(x)$ is the pdf that corresponds to $p_y(y)$ with respect to the change of the variable.

The books says it's because that observations falling in the range $(x, x + \delta x)$ will, for small values of $\delta x$, be transformed into the range $(y, y + \delta y)$.

How is this derived formally?

Update from Dilip Sarwate

The result holds only if $g$ is a strictly monotone increasing or decreasing function.

Some minor edit to L.V. Rao's answer $$$$P(Y\le y) = P(g(X)\le y)= \begin{cases} P(X\le g^{-1}(y)), & \text{if}\ g \text{ is monotonically increasing} \\ P(X\ge g^{-1}(y)), & \text{if}\ g \text{ is monotonically decreasing} \end{cases}$$$$ Therefore if $g$ is monotonically increasing $$F_{Y}(y)=F_{X}(g^{-1}(y))$$ $$f_{Y}(y)= f_{X}(g^{-1}(y))\cdot \frac{d}{dy}g^{-1}(y)$$ if monotonically decreasing $$F_{Y}(y)=1-F_{X}(g^{-1}(y))$$ $$f_{Y}(y)=- f_{X}(g^{-1}(y))\cdot \frac{d}{dy}g^{-1}(y)$$ $$\therefore f_{Y}(y) = f_{X}(g^{-1}(y)) \cdot \left | \frac{d}{dy}g^{-1}(y) \right |$$

• The result holds only if $g$ is a strictly monotone increasing or decreasing function. Draw a graph of $g$ and puzzle it out using the basic idea behind the definition of the derivative (not the formal definition with epsilon and delta). Also, there is an answer by @whuber on this site which spells out the details; that is, this should be closed as a duplicate. Commented Oct 11, 2016 at 15:32
• Your book's explanation is reminiscent of the one I offered at stats.stackexchange.com/a/14490/919. I also posted a general algebraic method at stats.stackexchange.com/a/101298/919 and a geometric explanation at stats.stackexchange.com/a/4223/919.
– whuber
Commented Oct 11, 2016 at 16:30
• @DilipSarwate thanks for your explanation, I think I understand the intuition, but I'm more interested in how it can be derived using the existing rules and theorems :) Commented Oct 12, 2016 at 6:18

Suppose $$X$$ is a continuous random variable with pdf $$f$$. Let $$Y=g(X)$$, where $$g$$ is a monotonic function.
The function $$g$$ could be either monotonically increasing or monotonically decreasing. If $$g$$ were monotonically increasing, then the pdf of $$Y$$ is obtained as follows: $$\begin{eqnarray*} P(Y\le y) &=& P(g(X)\le y)\\ &=& P(X\le g^{-1}(y))\\ or\;\;F_{Y}(y)&=& F_{X}(g^{-1}(y)),\quad \mbox{by the definition of CDF}\\ \end{eqnarray*}$$ If $$g$$ instead were monotonically decreasing, then we would have to swap the inequality signs, since $$g^{-1}$$ is also monotically decreasing (see here): $$\begin{eqnarray*} P(Y\le y) &=& P(g(X)\le y)\\ &=& P(X\ge g^{-1}(y))\\ or\;\;F_{Y}(y)&=& 1-F_{X}(g^{-1}(y)),\quad \mbox{by the definition of CDF}\\ \end{eqnarray*}$$
By differentiating the CDFs on both sides w.r.t. $$y$$ and using the chain rule, we get the pdf of $$Y$$. If the function $$g$$ is monotonically increasing, then the pdf of $$Y$$ is given by $$\begin{equation*} f_{Y}(y)= f_{X}(g^{-1}(y))\cdot \frac{d}{dy}g^{-1}(y) \end{equation*}$$ and other hand, if it is monotonically decreasing, then the pdf of $$Y$$ is given by $$\begin{equation*} f_{Y}(y)= - f_{X}(g^{-1}(y))\cdot \frac{d}{dy}g^{-1}(y) \end{equation*}$$ Since $$\left|\frac{d}{dy}g^{-1}(y)\right| = -\frac{d}{dy}g^{-1}(y)$$ if $$\frac{d}{dy}g^{-1}(y) \le 0$$ (which will be the case if $$g$$ is monotonically decreasing) and $$\left|\frac{d}{dy}g^{-1}(y)\right| = \frac{d}{dy}g^{-1}(y)$$ if $$\frac{d}{dy}g^{-1}(y)\ge 0$$ (which will be the case if $$g$$ is monotonically increasing), then the above two equations can be combined into a single equation: $$\begin{equation*} \therefore f_{Y}(y) = f_{X}(g^{-1}(y))\cdot \left|\frac{d}{dy}g^{-1}(y)\right| \end{equation*}$$
• @Chris The Jacobian of $g^{-1}$ is not necessarily a constant function, so it can be >1 in some places and <1 in others. Commented Jan 19, 2019 at 19:33
• I believe the above derivation is incorrect. When $g(.)$ is monotonically decreasing, $g(X) \le y \implies X \ge g^{-1}(y)$. The minus sign does not magically appear. Commented Feb 21, 2020 at 20:31