# Why is the delta method defined the way it is?

The delta method begins with the assumption of $$\sqrt{n} \left[X_n - \theta\right] \stackrel{D}{\to} \mathcal{N}(0, \sigma^2)$$. Why is this? Wouldn't it make more sense to start in the more familiar arrangement of $$X_n \stackrel{D}{\to} \mathcal{N}\left(\theta, \frac{\sigma^2}{n} \right)$$ ? Or even better, replace $$\theta$$ with $$\mu$$. Now it's a normal distribution in terms of mean and standard error.

I have the same question for the conclusion. Wouldn't the conclusion of $$g(X_n) \stackrel{D}{\to} \mathcal{N}\left(g(\theta),\frac{\sigma^2[g'(\theta)]^2}{n}\right)$$ make more sense to more readers for the same reason?

Is my math wrong? Is there some history of the delta method that dictates the original form? Thanks!

There are a few misconceptions to clear up first.

First, to clear up the biggest misconception. The final expression you gave has the limiting distribution also varying with $$n$$; this cannot be! Your limiting distribution must be one that is independent of $$n$$. Of course, you can say that $$g(x_n)$$ is approximately distributed as the normal distribution you provide. Here, no asymptotic statements are being made, so we don't have the undesirable situation where our limit varies with $$n$$ as well. In fact, one could say when we're talking about approximate distributions, we want our distribution on the RHS to depend on $$n$$, but, that's besides the point.

Secondly, the (univariate) delta method begins with that assumption you give because we want to know the limiting distribution of

$$\frac{\sqrt{n}(g(X_n) - g(\theta))}{\sigma^2}$$.

If we assume that the limiting distribution of $$\sqrt{n}(X_n - \theta)$$ is $$\mathcal{N}(0, \sigma^2)$$, then the limiting distribution of the desired expression is normal, just with different parameters. These parameters are given via a first-order Taylor expansion of $$g(X_n)$$ about $$\theta$$, which is why we have the added assumption that $$g'$$ is continuous at $$\theta$$.

Notice, though that $$\theta$$ is any parameter, and $$X_n$$ is a sequence of estimators for that parameter. $$\theta$$ need not be any mean, say.

• Yes, $\mu = E(X)$ was my assumption. Commented Nov 25, 2020 at 0:30
• So, in short, we don't write it that way, because we don't want to create a normal distribution (approximate, asymptotic, or otherwise) that is dependent on sample size? Commented Nov 25, 2020 at 0:33
• Almost! If the normal distribution is the limit of a sequence of random variables converging in distribution, it needs to be fixed with respect to the variable that is "going to infinity". You can certainly have an approximate distribution that depends on sample size, though. Commented Nov 25, 2020 at 1:40
• Think in terms of sequences of real numbers. I can have a sequence of real numbers $a_n$ that converges to a fixed real number $a$, but the notion of a sequence of real numbers $a_n$ "converging to" a sequence of real numbers $b_n$ is nonsense! You can certainly say that $a_n$ "for large n" is approximately $b_n$, though. Analogous statements are made in the previous comment. Commented Nov 25, 2020 at 1:45