3
$\begingroup$

If a random variable $W$ is Normally distributed, then $\exp(W)$ is Log-Normally distributed.

However, the pdfs of these two random variables differ by a factor of $\exp(W)^{-1}$.

The Normal pdf for $W$ is

$$P(w \in W) = \frac{1}{\sigma\sqrt{2\pi}}\, e^{-\frac{(w - \mu)^2}{2 \sigma^2}}$$

but the Log-Normal pdf for $\exp(W)$ is

$$P(\exp(w) \in \exp(W)) = \frac{1}{\exp(w)\sigma\sqrt{2\pi}}\ e^{-\frac{\left(w-\mu\right)^2}{2\sigma^2}}$$

Why is this the case? Shouldn't they be equivalent?

|cite|improve this question
$\endgroup$
  • 3
    $\begingroup$ The appearance of this factor for general transformations of random variables is explained at stats.stackexchange.com/questions/14483. Reading that might help you appreciate, too, that "$P(w\in W)$" makes no sense. ($W$ is a random variable, not a set; and if you replace "$\in$" by "$=$," which does make sense, you will obtain $P(w=W)=0$: see stats.stackexchange.com/questions/99935.) It suggests you are thinking of the PDF in the same way one ought to think of the cumulative distribution function. The lognormal CDF has no extra factor in it, exactly as you might suppose. $\endgroup$ – whuber♦ May 25 '14 at 18:17
2
$\begingroup$

I'd like to provide a simple and straightforward demonstration to where this factor comes from.

Let $p(x)$ be the PDF of a normal distribution $\mathcal{N}\left(\mu,\sigma^2\right)$. Then, of course:

$$\int\limits_{-\infty}^{+\infty} p(x)dx = \frac{1}{\sqrt{2\pi}\sigma} {\rm e}^ {-\frac{\left( x-\mu \right)^2}{2\sigma^2} } {\rm d}x = 1$$

If you now make a substitution $x = \log y$, i.e. $x$ is distributed normally, and $y$ has a log-normal distribution, you'll get

$$\int\limits_{-\infty}^{+\infty} p(\log y){\rm d} (\log y) = 1 = \frac{1}{\sqrt{2\pi}\sigma} {\rm e}^ {-\frac{\left( \log y-\mu \right)^2}{2\sigma^2} } \boxed{{\rm d}(\log y)} $$

and the boxed term is $\frac{{\rm d}y}{y}$, showing where does the $\frac{1}{y}$ come from. So that the PDF of $y$ is

$$\frac{1}{\sqrt{2\pi}\sigma y} e^ {-\frac{\left( \log y-\mu \right)^2}{2\sigma^2} }$$

|cite|improve this answer
$\endgroup$
3
$\begingroup$

The factor you speak of is simply the Jacobian of the transformation, the determinant of the transformation from one variable (or set of variables) to another.

It has nothing in particular to do with normality and everything to do with the ($\exp$) transformation.

It's a straightforward calculation:

If $Y=\exp(X)$ and the cdf of $X$ is $F_X$, then

$P(Y\leq y) = P(\exp(X)\leq y) = P(X\leq \log(y)) = F_X(\log(y))$

Hence the pdf of $Y$ is $\frac{\partial}{\partial y} F_X(\log(y)) = f_X(\log(y))\cdot\frac{1}{y}\,,\quad$ (by the chain rule for differentiation).

More generally, if $Y$ is some different monotonic increasing function of $X$, $Y=g(X)$, then

$P(Y\leq y) = P(g(X)\leq y) = P(X\leq g^{-1}(y)) = F_X(g^{-1}(y))\,$

and so the pdf of $Y$ is $\frac{\partial}{\partial y} F_X(g^{-1}(y)) = f_X(g^{-1}(y))\cdot\frac{\partial}{\partial y}g^{-1}(y)$.

A similar relationship (up to a sign change) holds for monotonic decreasing transformations.

See here, which deals with both forms.

--

You can't write your second density that way, because you're dealing with the density for the transformed variable; if you're going to substitute $w$ into that density you need the Jacobian for the inverse transform (the one back to $w$), which takes you straight back to the density for $w$ you started with.

The first post whuber links to in his comment gives some additional intuition for what's going on.

|cite|improve this answer
$\endgroup$
  • $\begingroup$ Thanks for your explanation. What is the correct way to write the second density? If I have an observation $x$, how do I calculate $P(x)$ using the normal and log-normal pdfs? $\endgroup$ – rhombidodecahedron May 26 '14 at 0:15
  • $\begingroup$ If $W$ is normal and $Y = exp(W)$ then $f_Y(y) = \frac{1}{y\sigma\sqrt{2\pi}}\ \exp(-\frac{\left(\log(y)-\mu\right)^2}{2\sigma^2})\,,\quad y>0$. See here. $\endgroup$ – Glen_b May 26 '14 at 0:27
  • $\begingroup$ Right, but that's basically the same expression I had originally. But if you calculate it using the normal pdf, you get a different result. Do you see what I'm getting at here? $\endgroup$ – rhombidodecahedron May 26 '14 at 3:38
  • 1
    $\begingroup$ I understand what you're saying, but when you say "basically the same", that's where you're going astray. Density doesn't work that way, since the transformation "stretches the density", and you have to account for how $dw$ transforms to $dy$ (literally, $dw=\frac{1}{y} dy$ in your problem). You can't simply replace variables in the density like that, you need also to take the element $dw=\frac{1}{y} dy$ across. It (that replacement of $y$ by $\exp(w)$ when $Y=e^W$) can, however be done with probabilities of events, which is why you can do it with the CDF. $\endgroup$ – Glen_b May 26 '14 at 3:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.