# Delta method vs actual expectation

If $$x \sim N(\mu,\sigma^2)$$, then by first principles,

$$\mathbb{E}(e^x) = e^{\mu + \sigma^2 / 2}.$$

I am trying to figure out where the "Delta method" is wrong here: If $$(x-\mu) \sim N(0,\sigma^2)$$, then

$$(f(x)-f(\mu)) \sim N(0,\sigma^2 f'(\mu)^2).$$

The Delta method using $$f(x) = e^x$$ will imply

$$(e^x-e^\mu) \sim N(0,\sigma^2 (e^\mu)^2 ),$$

which implies

$$\mathbb{E}(e^x) = e^\mu.$$

• Why do you think $f(x) - f(\mu)$ will have a Normal distribution, much less one centered at zero? Commented May 16 at 21:15
• To be a little strict, delta method is not used to calculate the exact expectation of $f(x)$, it is an asymptotic tool. Commented May 16 at 22:45

To see what is happening, let's go back to the first principles of the Delta method. The Delta method is based upon the Taylor expansion of a function $$f(x)$$ around the value $$f(\mu)$$, where $$\mu$$ is the mean of the function:

$$f(x) = f(\mu) + (x-\mu)f'(\mu) + {1\over 2}(x-\mu)^2f''(\mu) + \dots$$

Taking the expectation and noting that $$\mathrm{E}(x-\mu) = 0$$ by definition of $$\mu$$ gives us:

$$\mathrm{E}f(x) = f(\mu) + {1\over 2}\mathrm{E}(x-\mu)^2f''(\mu) + \dots$$

Dropping all but the first two terms of the expansion and substituting $$\sigma^2 = \mathrm{E}(x-\mu)^2$$ results in:

$$\mathrm{E}f(x) \approx f(\mu) + {1\over 2}\sigma^2f''(\mu)$$

Now for this specific case. Setting $$f(x) = e^x$$ and substituting gives:

$$\mathrm{E}e^x \approx e^{\mu} + {1\over 2}\sigma^2e^{\mu}$$

Rearranging terms slightly:

$$\mathrm{E}e^x \approx e^{\mu}\left(1 + {1\over 2}\sigma^2\right)$$

Now, let's consider the Maclaurin expansion (Taylor expansion around zero) of $$e^x$$:

$$e^x = 1 + x + {x^2\over 2} + {x^3\over 6} + \dots$$

If we substitute $${1\over 2}\sigma^2$$ for $$x$$ in the above, we can readily see that the term $$\left(1 + {1\over 2}\sigma^2\right)$$ which multiplies $$e^{\mu}$$ above corresponds to the first two terms of the Maclaurin expansion of $$\exp\{{1\over 2}\sigma^2\}$$. If we were to use the full series, which we can do because we know it converges, we would get:

$$\mathrm{E}e^x = e^{\mu}e^{{1\over 2}\sigma^2} = e^{\mu + {1\over 2}\sigma^2}$$

which is, as you've observed, correct.

So what has happened here is that we have truncated an infinite series as part of our application of the Delta method, and the difference between the exact result and the approximate result of the Delta method is due to that truncation.

The delta method gives you a first-order approximation, one that is based on the slope $$f'$$ only at a single point. In general, it gives the right answer under asymptotic conditions when $$\sigma^2\to 0$$ but is only approximate for fixed $$\sigma^2$$

In your case you can see this happening: the approximation is good when $$\sigma^2$$ is small and bad when $$\sigma^2$$ is large. You often use the delta-method when $$\sigma^2$$ is a multiple of $$1/n$$, so $$e^\mu$$ and $$e^{\mu+\sigma^2/2}$$ are actually pretty close to each other.