Intuitive explanation for density of transformed variable?

Question

Suppose $X$ is a random variable with pdf $f_X(x)$. Then the random variable $Y=X^2$ has the pdf

$f_Y(y)=\left\{\begin{array}{ll}\frac{1}{2\sqrt{y}}\left(f_X(\sqrt{y})+f_X(-\sqrt{y})\right) & y \ge 0 \\ 0 & y \lt 0\end{array}\right.$

I understand the calculus behind this. But I'm trying to think of a way to explain it to someone who doesn't know calculus. In particular, I'm trying to explain why the factor $\frac{1}{\sqrt{y}}$ appears out front. I'll take a stab at it:

Suppose $X$ has a Gaussian distribution. Almost all the weight of its pdf is between the values, say, -3 and 3. But that maps to 0 to 9 for $Y$. So, the heavy weight in the pdf for $X$ has been extended across a wider range of values in the transformation to $Y$. Thus, for $f_Y(y)$ to be a true pdf the extra heavy weight must be downweighted by the multiplicative factor $\frac{1}{\sqrt{y}}$

How does that sound?

If anyone can provide a better explanation of their own or link to one in a document or textbook I'd greatly appreciate it. I find this variable transformation example in several intro mathematical probability/stats books. But I never find an intuitive explanation with it :(

Answer 1

PDFs represent probability by means of area. It therefore helps to express a PDF in a way that reminds us that area equals height times base. Initially the height at $x$ is given by $f_X(x)$. The base is the infinitesimal segment $dx$, whence the distribution (as opposed to the distribution function) is really the differential form

$$\text{pdf}_X(x) = f_X(x) dx.$$

When we re-express $x$ in terms of $y = x^2$, the base segments $dx$ get stretched (or squeezed): by squaring both ends of the interval from $x$ to $x + dx$ we see that the base of the $y$ area must be an interval of length

$$dy = (x + dx)^2 - x^2 = 2 x dx + (dx)^2.$$

Because the product of two infinitesimals is negligible compared to the infinitesimals themselves, we conclude

$$dy = 2 x dx, \text{ whence }dx = \frac{dy}{2x} = \frac{dy}{2\sqrt{y}}.$$

Having established this, the calculation is trivial because we just plug in the new height and the new width:

$$f_X(x) dx = f_X(\sqrt{y}) \frac{dy}{2\sqrt{y}} = \text{pdf}_Y(y).$$

Because the base, in terms of $y$, is $dy$, whatever multiplies it must be the height:

$$\frac{1}{2\sqrt{y}}f_X(\sqrt{y}) = f_Y(y).$$

This is effectively a conservation of area (=probability) law.

This graphic accurately shows narrow (almost infinitesimal) pieces of two PDFs related by $y=x^2$. Probabilities are represented by the shaded areas. Due to the squeezing of the interval $[0.32, 0.45]$ via squaring, the height of the red region ($y$, at the left) has to be proportionally expanded to match the area of the blue region ($x$, at the right).

Answer 2

How about, if I manufacture objects that are ALWAYS square and I know the distribution of the side lengths of the squares; what can I say about the distribution of the areas of the squares?

In particular, if I know the distribution of a random variable $X$, what can I say about $Y = X^{2}$ ? One thing that you can say is

$$\eqalign{ F_{Y} (c) & = & P( Y \le c ) \\ & = & P( X^{2} \le c ) \\ & = & P ( - \sqrt{c} \le X \le \sqrt{c}) \\ & = & F_{X}( \sqrt{c} ) - F_{X}( - \sqrt{c} ). \\ }$$

So a relationship is established between the CDF of $Y$ and CDF of $X$; what is the relationship between their PDFs? We need Calculus for that. Taking the derivatives of both sides gives you the results you wanted.