I have been solving one problem and there is something unclear to me in the solutions.

Namely, let's consider a probability density $p_x(x)$ defined over a continuous variable $x$, and suppose that we make a nonlinear change of variable using $x = g(y)$, so that the density transforms according to (1.27). $$\begin{align} p_y(y) &= p_x(x) \left| \dfrac{dx}{dy} \right| \\ &= p_x(g(y)) |g'(y)| \tag{1.27} \end{align}$$

An example of this: Let's take a Gaussian distribution $p_x(x)$ over $x$ with mean $\mu = 6$ and standard deviation $\sigma = 1$. Next we draw a sample of $N = 50,000$ points from this distribution.

Now let's consider a non-linear change of variables from $x$ to $y$ gives by

$$x = g(y) = \ln(y) - \ln(1 - y) + 5 \tag{5}$$

The inverse of this function is given by

$$y = g^{-1}(x) = \dfrac{1}{1 + \exp(-x + 5)} \ \tag{6}$$

which is a logistic sigmoid function.

Now, in order to calculate $p_y(y)$ we can use the (1.27) defined above, by basically taking into account (5) and computing the Jacobian, i.e. $|g'(y)|$.

The unclear part: The author claims that we can confirm whether we did the non-linear transformation correctly by taking out sample of 50,000 values of $x$, evaluate the corresponding values of $y$ using (6) and plot this, and this would match the transformation given by 1.27.

So my question now: how did we avoid computing the Jacobian by using the inverse i.e (6) instead, and why don't we always do this instead, i.e. just compute the inverse, and not do a full transformation according to 1.27?

Thanks in advance!


1 Answer 1


The Jacobian is avoided via simulation since the density for $y$ is defined as \begin{eqnarray*} p_y(t) &=& \underset{\substack{\Delta t \rightarrow 0 \\ n \rightarrow \infty}}{\mbox{lim}}\frac{\frac{1}{n} \sum_{i=1}^n \mbox{I} \left[t \lt g^{-1}(x_i) \lt t + \Delta t \right]}{\Delta t} \\ &=& \underset{\substack{\Delta t \rightarrow 0 \\ n \rightarrow \infty}}{\mbox{lim}}\frac{\frac{1}{n} \sum_{i=1}^n \mbox{I} \left[t \lt y_i \lt t + \Delta t \right]}{\Delta t}\\ &=& \underset{\Delta t \rightarrow 0}{\mbox{lim}} \frac{\mbox{Pr} \left[ t \lt y \lt t + \Delta t\right]}{\Delta t} \\ &=& \underset{\Delta t \rightarrow 0}{\mbox{lim}} \frac{F_y(t+\Delta t) - F_y(t)}{\Delta t}, \end{eqnarray*} where $\mbox{I} \left[\cdot\right]$ denotes the indicator function and $F_y (\cdot)$ denotes the CDF of $y$. Since we have 50,000 simulations, it is safe to assume that $n$ is large enough for the limiting argument to hold. In other words, \begin{eqnarray*} p_y(t) & \approx & \underset{\substack{\Delta t \rightarrow 0}}{\mbox{lim}}\frac{\frac{1}{50,000} \sum_{i=1}^{50,000} \mbox{I} \left[t \lt g^{-1}(x_i) \lt t + \Delta t \right]}{\Delta t}. \end{eqnarray*}

Using simulation is undesirable in the sense that it does not give a closed-form solution for $p_y(t)$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.