6
$\begingroup$

To understand what I mean, let's use two well-known distributions: the normal and lognormal ones.

From the dataset point of view, if you take normally-distributed data and take their exponential, you get lognormally-distributed data (norm -exp-> lognorm).

So the "idea" was: why not using the PDF directy and transform the normal PDF to the lognormal one by taking the exponential of its x-axis. The reasoning was that this x-axis represents my data: taking the exponential of these data or the exponential of the x-axis of the corresponding PDF should do the same.

But I did not even bother to do it: mathematically, this is not how one can transform the normal PDF to the lognormal one!

What I don't understand is why just taking the exponential of the x-axis of the PDF does not work. I mean, not mathematically (I get that there is a $1/x$ in the lognormal PDF!), but what is conceptually wrong with "just stretching" the axis?

$\endgroup$
3
  • $\begingroup$ Stretching one axis does not preserve the essential, defining property of a PDF that it represent probability by means of area. For that property to be preserved, all stretching along the $x$-axis must be compensated for by an amount of compression along the $y$ axis that is inversely proportional to the stretching. If instead you were to examine the CDF, which represents probability by height, then stretching the $x$-axis alone is sufficient (assuming the transformation is a one-to-one increasing function, of course). $\endgroup$
    – whuber
    Jul 12, 2016 at 13:16
  • $\begingroup$ stats.stackexchange.com/questions/14483 is essentially the same question: you might be interested in the discussion there. My answer in that thread illustrates the preceding comment. $\endgroup$
    – whuber
    Jul 12, 2016 at 13:18
  • 1
    $\begingroup$ Yes thanks for the pointer! This is the kind of problem which can be expressed/viewed in multiple different ways so it is not really easy to find previous answers. $\endgroup$
    – JohnW
    Jul 12, 2016 at 13:49

1 Answer 1

6
$\begingroup$

You're heading in the right direction with your thoughts on considering the cdf.

Consider some random variable, $X$ with cdf $F_X(x)$ and density $f_X(x)$. To make things simple, consider applying some monotonic increasing transformation, $t$ on $X$, giving $Y=t(X)$. The new variable $Y$ has cdf $F_Y(y)$ and density $f_Y(y)$. Then:

$F_Y(y) = P(Y\leq y) = P(t(X)\leq y) = P(X\leq t^{-1}(y)) = F_X(t^{-1}(y))$

(By plotting $F_X(t^{-1}(y))$ against $y$ , this has the "stretching" effect on the x-axis you mentioned - the values on the vertical axis are unchanged but are shifted on the horizontal axis.)

Now we can see where that $\frac{1}{x}$ term came from in the lognormal pdf.
Recall we had:

$F_Y(y) = F_X(t^{-1}(y))$

So

$f_Y(y) = \frac{d}{dy} F_X(t^{-1}(y)) = f_X(t^{-1}(y))\cdot \frac{d}{dy}t^{-1}(y)$

A similar result can be derived for monotonic decreasing transformations, yielding the more general result for invertible transformations:

$f_Y(y) = \frac{d}{dy} F_X(t^{-1}(y)) = f_X(t^{-1}(y))\cdot |\frac{d}{dy}t^{-1}(y)|$

When $t$ is the $\exp$ function, $t^{-1}$ is the log, which has the reciprocal as its derivative.

So you do that axis transformation you thought about, but you then have an additional factor, the Jacobian of the transformation, which changes the height. So far it's quite clear that we must have that term when we go to the pdf from the CDF.

But we can also explain more directly why we need it:

Loosely, note that if you have a very small interval $[x,x+\delta x)$ for which $f$ is effectively constant (so the area is effectively $f(x)\,\delta x$), if you stretch the axis by transforming it as for the cdf, the total area in the transformed small interval is changed by the stretching, but the probability of being in the interval is unchanged. So to preserve the probability represented by the small area, you need to "undo" the impact of the stretching on the small area so that it still represents the probability. The area is kept the same by modifying the height. (This is what the Jacobian does -- preserve small areas.)

![enter image description here

Note that dividing in our example by $t'(x)=\exp(x)$ is in that case the same as dividing by $y$, which is the scaling factor we get from the Jacobian calculation above for $t(x)=\exp(x)$.

$\endgroup$
1
  • $\begingroup$ Brilliant! The graphs are also a nice addition. $\endgroup$
    – JohnW
    Jul 12, 2016 at 15:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.