44
$\begingroup$

In the book pattern recognition and machine learning (formula 1.27), it gives

$$p_y(y)=p_x(x) \left | \frac{d x}{d y} \right |=p_x(g(y)) | g'(y) |$$ where $x=g(y)$, $p_x(x)$ is the pdf that corresponds to $p_y(y)$ with respect to the change of the variable.

The books says it's because that observations falling in the range $(x, x + \delta x)$ will, for small values of $\delta x$, be transformed into the range $(y, y + \delta y)$.

How is this derived formally?


Update from Dilip Sarwate

The result holds only if $g$ is a strictly monotone increasing or decreasing function.


Some minor edit to L.V. Rao's answer $$ \begin{equation} P(Y\le y) = P(g(X)\le y)= \begin{cases} P(X\le g^{-1}(y)), & \text{if}\ g \text{ is monotonically increasing} \\ P(X\ge g^{-1}(y)), & \text{if}\ g \text{ is monotonically decreasing} \end{cases} \end{equation}$$ Therefore if $g$ is monotonically increasing $$F_{Y}(y)=F_{X}(g^{-1}(y))$$ $$f_{Y}(y)= f_{X}(g^{-1}(y))\cdot \frac{d}{dy}g^{-1}(y)$$ if monotonically decreasing $$F_{Y}(y)=1-F_{X}(g^{-1}(y))$$ $$f_{Y}(y)=- f_{X}(g^{-1}(y))\cdot \frac{d}{dy}g^{-1}(y)$$ $$\therefore f_{Y}(y) = f_{X}(g^{-1}(y)) \cdot \left | \frac{d}{dy}g^{-1}(y) \right |$$

$\endgroup$
3
  • 2
    $\begingroup$ The result holds only if $g$ is a strictly monotone increasing or decreasing function. Draw a graph of $g$ and puzzle it out using the basic idea behind the definition of the derivative (not the formal definition with epsilon and delta). Also, there is an answer by @whuber on this site which spells out the details; that is, this should be closed as a duplicate. $\endgroup$ Oct 11, 2016 at 15:32
  • 1
    $\begingroup$ Your book's explanation is reminiscent of the one I offered at stats.stackexchange.com/a/14490/919. I also posted a general algebraic method at stats.stackexchange.com/a/101298/919 and a geometric explanation at stats.stackexchange.com/a/4223/919. $\endgroup$
    – whuber
    Oct 11, 2016 at 16:30
  • 1
    $\begingroup$ @DilipSarwate thanks for your explanation, I think I understand the intuition, but I'm more interested in how it can be derived using the existing rules and theorems :) $\endgroup$
    – dontloo
    Oct 12, 2016 at 6:18

1 Answer 1

49
$\begingroup$

Suppose $X$ is a continuous random variable with pdf $f$. Let $Y=g(X)$, where $g$ is a monotonic function.

The function $g$ could be either monotonically increasing or monotonically decreasing. If $g$ were monotonically increasing, then the pdf of $Y$ is obtained as follows: \begin{eqnarray*} P(Y\le y) &=& P(g(X)\le y)\\ &=& P(X\le g^{-1}(y))\\ or\;\;F_{Y}(y)&=& F_{X}(g^{-1}(y)),\quad \mbox{by the definition of CDF}\\ \end{eqnarray*} If $g$ instead were monotonically decreasing, then we would have to swap the inequality signs, since $g^{-1}$ is also monotically decreasing (see here): \begin{eqnarray*} P(Y\le y) &=& P(g(X)\le y)\\ &=& P(X\ge g^{-1}(y))\\ or\;\;F_{Y}(y)&=& 1-F_{X}(g^{-1}(y)),\quad \mbox{by the definition of CDF}\\ \end{eqnarray*}

By differentiating the CDFs on both sides w.r.t. $y$ and using the chain rule, we get the pdf of $Y$. If the function $g$ is monotonically increasing, then the pdf of $Y$ is given by \begin{equation*} f_{Y}(y)= f_{X}(g^{-1}(y))\cdot \frac{d}{dy}g^{-1}(y) \end{equation*} and other hand, if it is monotonically decreasing, then the pdf of $Y$ is given by \begin{equation*} f_{Y}(y)= - f_{X}(g^{-1}(y))\cdot \frac{d}{dy}g^{-1}(y) \end{equation*} Since $\left|\frac{d}{dy}g^{-1}(y)\right| = -\frac{d}{dy}g^{-1}(y)$ if $\frac{d}{dy}g^{-1}(y) \le 0$ (which will be the case if $g$ is monotonically decreasing) and $\left|\frac{d}{dy}g^{-1}(y)\right| = \frac{d}{dy}g^{-1}(y)$ if $\frac{d}{dy}g^{-1}(y)\ge 0$ (which will be the case if $g$ is monotonically increasing), then the above two equations can be combined into a single equation: \begin{equation*} \therefore f_{Y}(y) = f_{X}(g^{-1}(y))\cdot \left|\frac{d}{dy}g^{-1}(y)\right| \end{equation*}

$\endgroup$
6
  • $\begingroup$ But as the integral over fx must sum to 1 and fy is a scaled version of fx, doesn't that mean fy is not a proper pdf, unless the jacobian in the abs() is 1 or -1? $\endgroup$
    – Chris
    Oct 11, 2018 at 18:18
  • 1
    $\begingroup$ @Chris The Jacobian of $g^{-1}$ is not necessarily a constant function, so it can be >1 in some places and <1 in others. $\endgroup$ Jan 19, 2019 at 19:33
  • 2
    $\begingroup$ I believe the above derivation is incorrect. When $g(.)$ is monotonically decreasing, $g(X) \le y \implies X \ge g^{-1}(y)$. The minus sign does not magically appear. $\endgroup$ Feb 21, 2020 at 20:31
  • 1
    $\begingroup$ The minus sign comes from the fact that the inequality is switched for monotonically decreasing transformations $\endgroup$
    – Sebastian
    Mar 8, 2020 at 0:05
  • $\begingroup$ Where is the justification/explanation of the bit about monotonicity $\endgroup$
    – user3180
    Jul 4, 2021 at 8:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.