Why is the delta method defined the way it is?

Question

The delta method begins with the assumption of $\sqrt{n} \left[X_n - \theta\right] \stackrel{D}{\to} \mathcal{N}(0, \sigma^2)$. Why is this? Wouldn't it make more sense to start in the more familiar arrangement of $X_n \stackrel{D}{\to} \mathcal{N}\left(\theta, \frac{\sigma^2}{n} \right)$ ? Or even better, replace $\theta$ with $\mu$. Now it's a normal distribution in terms of mean and standard error.

I have the same question for the conclusion. Wouldn't the conclusion of $g(X_n) \stackrel{D}{\to} \mathcal{N}\left(g(\theta),\frac{\sigma^2[g'(\theta)]^2}{n}\right)$ make more sense to more readers for the same reason?

Is my math wrong? Is there some history of the delta method that dictates the original form? Thanks!

Answer 1

There are a few misconceptions to clear up first.

First, to clear up the biggest misconception. The final expression you gave has the limiting distribution also varying with $n$; this cannot be! Your limiting distribution must be one that is independent of $n$. Of course, you can say that $g(x_n)$ is approximately distributed as the normal distribution you provide. Here, no asymptotic statements are being made, so we don't have the undesirable situation where our limit varies with $n$ as well. In fact, one could say when we're talking about approximate distributions, we want our distribution on the RHS to depend on $n$, but, that's besides the point.

Secondly, the (univariate) delta method begins with that assumption you give because we want to know the limiting distribution of

$$\frac{\sqrt{n}(g(X_n) - g(\theta))}{\sigma^2}$$.

If we assume that the limiting distribution of $\sqrt{n}(X_n - \theta)$ is $\mathcal{N}(0, \sigma^2)$, then the limiting distribution of the desired expression is normal, just with different parameters. These parameters are given via a first-order Taylor expansion of $g(X_n)$ about $\theta$, which is why we have the added assumption that $g'$ is continuous at $\theta$.

Notice, though that $\theta$ is any parameter, and $X_n$ is a sequence of estimators for that parameter. $\theta$ need not be any mean, say.