How is the asymptotic justification of the "linearization by influence function method" for surveys established?

Question

The survey R package recently adopted the "linearization by influence function" method of estimating covariances between domain estimates. The central paper justifying this method is Deville (1999). I'm trying to understand the main asymptotic claim made in the paper and some confusing aspects of the proof.

The result is summarized concisely by Deville and others in this 2009 Biometrika paper:

In Deville's approach, a population parameter of interest $\Phi$ can be written as a functional $T$ with respect to a finite and discrete measure $M$, namely $\Phi=T(M)$. The substitution estimator $\hat{\Phi}=T(\hat{M})$ is the functional $T$ of a random measure $\hat{M}$ that is associated with sampling weights $w_{k}, k \in U$, and is 'close' to $M$. Suppose that $T$ is homogeneous of degree $\alpha$, so that $T(r M)=r^{\alpha} T(M)$, and $\lim _{N \rightarrow \infty} N^{-\alpha} T(M)<\infty$. Under broad assumptions, Deville shows that $$ \begin{aligned} \sqrt{n} N^{-\alpha}\{T(\hat{M})-T(M)\} &=\sqrt{n} N^{-\alpha} \int I_{T}(M, z) d(\hat{M}-M)(z)+o_{p}(1) \\ &=\sqrt{n} N^{-\alpha} \sum_{k=1}^{N} u_{k}\left(w_{k}-1\right)+o_{p}(1) \end{aligned} $$ The linearized variables $u_{k}$ are the influence functions $I_{T}\left(M, z_{k}\right)$, where $z_{k}$ is the value of the variable of interest for the $k$ th unit.

This exact equation doesn't show up in Deville's 1999 paper as far as I can see. Instead, there is a tantalizingly similar-looking result on page 6.

Result: Under broad assumptions, the substitution estimation of a functional $T(M)$ is linearizable. A linearized variable is $z_{k}=I T\left(M ; x_{k}\right)$ where $I T$ is the influence function of $T$ in $M$.

Proof of the result: Let us provide the space of measurements on $\boldsymbol{R}^{q}$ with a metric $d$ accounting for the convergence: $d\left(M_{1}, M_{2}\right) \rightarrow 0$ if and only if $N^{-1}\left(\int y d M_{1}-\int y d M_{2}\right) \rightarrow 0$ for any variable of interest $y$. The asymptotic postulates mean that $d(\hat{M} / N, M / N)$ tends towards zero. We can visibly ensure that $d(\hat{M} / N, M / N)$ is $O_{p}(1 / \sqrt{n})$ according to the third postulate. Now, let us assume that $T$ can be derived in accordance with Fréchet, i.e., for any direction of the increase, in the space of "useful" measures provided with the abovementioned metric. Thus we have: $$ N^{-\alpha}(T(\hat{M})-T(M))=\frac{1}{N} \sum_{U} z_{k}\left(w_{k}-1\right)+o\left(d\left(\frac{\hat{M}}{M}, \frac{M}{N}\right)\right) $$ The result is that: $$ \sqrt{n} N^{-\alpha}(T(\hat{M})-T(M))=\frac{\sqrt{n}}{N} \sum_{U} z_{k}\left(w_{k}-1\right)+o_{p}(1) . $$

However, this result in the original 1999 paper is not the same result as the 2009 Biometrika paper. Why does the right-hand side of the Deville 1999 equation use $N^{-1}$, while the right-hand side of the quoted equation in the 2009 paper uses $N^{-\alpha}$?

The equation in Deville 1999 doesn't make sense to me. For example, the mean is a statistic of degree 0 and its influence function is $z_{k}=\frac{1}{N}\left(y_{k}-\bar{Y}\right)$, so with the Deville 1999 equation we would end up with the nonsensical result that $\sqrt{n}(\hat{\bar{Y}} - \bar{Y}) = \frac{\sqrt{n}}{N} \left[\hat{\bar{Y}} - \bar{Y}(\frac{\hat{N}}{N}) \right] + o_p(1)$.

And the proof seems to contain some hidden steps. How is that first equation in Deville 1999 derived? It seems to involve the following missing step, but it's not clear how even this equation would be established.

$$ N^{-\alpha}(T(\hat{M})-T(M))= N^{-1} \int I_{T}(M, z) d(\hat{M}-M)(z)+o\left(d\left(\frac{\hat{M}}{M}, \frac{M}{N}\right)\right) $$

Answer 1

So the Deville 1999 paper does have a typo there when it comes to $N^{-1}$ vs. $N^{-\alpha}$. A couple of subsequent papers contain clearer, more detailed proofs in their appendices: Goga, Deville, and Ruiz-Gazen, 2009, and Goga and Ruiz-Gazen 2013. This blog post I wrote contains a fairly detailed walk-through of the proof from the 2009 paper linked above. This StackExchange answer represents a condensed version of the blog post.

Assumptions of the theorem

The first set of assumptions (1 through 4) apply to the sample design and must apply to each variable of interest, $\mathcal{Z}$.

• Assumption 1. We assume that $\lim _{N \rightarrow \infty} N^{-1} n = f\in(0,1)$.

• Assumption 2 . We assume that $\lim _{N \rightarrow \infty} N^{-1} \int \mathcal{Z} d M$ exists, for any variable of interest $\mathcal{Z}$.

• Assumption 3. As $N \rightarrow \infty$, we have that $N^{-1}\left(\int \mathcal{Z} d \hat{M}-\int \mathcal{Z} d M\right) \rightarrow 0$ in probability, for any variable of interest $\mathcal{Z}$.

• Assumption 4. As $N \rightarrow \infty,\left\{\sqrt{n} N^{-1}\left(\int \mathcal{Z} d \hat{M}-\int \mathcal{Z} d M\right)\right\} \rightarrow N(0, \Sigma)$ in distribution, for any variable of interest $\mathcal{Z}$.

The second set of assumptions, 5 through 7, apply to the functional $T$, requiring it to be "smooth" in a certain sense and to be bounded in a certain sense.

• Assumption 5. We assume that $T$ is homogeneous, in that there exists a real number $\beta>0$ dependent on $T$ such that $T(r M)=r^{\beta} T(M)$ for any real $r>0$.

• Assumption 6. We assume that $\lim _{N \rightarrow \infty} N^{-\beta} T(M)<\infty$.

• Assumption 7. We assume that $T$ is Fréchet differentiable.

Statement of Theorem

Let $u_k$ denote the influence function for element $k$ in the population (i.e. if person $k$ has values $z_k$, then $u_k=IT(M;z_k)$), and let $v_k$ denote the sampling weight for person $k$ in the population. If person $k$ is in a selected sample, then $v_k=1/\pi_k$, where $\pi_k$ is their probability of inclusion in the sample. For persons in the population but not in the selected sample, $v_k=0$.

If Assumptions 1 through 7 hold, then:

$$ \begin{aligned}\frac{\sqrt{n}}{N^{\beta}}\{T(\hat{M})-T(M)\} &=\frac{\sqrt{n}}{N^{\beta}} \int IT(M ; z) d\left(\hat{M}-M\right)(z)+o_{p}(1) \\&=\frac{\sqrt{n}}{N^{\beta}} \left\{\sum_{k=1}^{N} u_{k}\left(v_{k}-1\right)\right\}+o_{p}(1) \end{aligned} $$

$$ \textit{and so the asymptotic variance of }\\T(\hat{M})\textit{ is equal to the variance of }\\ \sum_{k=1}^{N} u_{k}\left(v_{k}-1\right) $$

Proof

Below is a condensed version of that post:

From Assumptions 5 (homogeneity of $T$), we have that $N^{-\beta}T(M)=T(M/N)$. From Assumption 6, we have that $T(M/N)=N^{-\beta}T(M) < \infty$ for sufficiently large $N$. Taken together, this implies that for sufficiently large $N$, the following holds:

$$ \begin{aligned}{N^{-\beta}}\{T(\hat{M})-T(M)\} &= T(\frac{\hat{M}}{N}) - T(\frac{M}{N}) \\ \end{aligned} $$

Let us provide the space $(\mathbb{R}^p, M)$ with the metric $\tilde{d}$ satisfying $d(Q/N, M/N) \rightarrow 0$ if and only if $N^{-1} \{ \int Z dQ(z) - \int Z dM(z) \} \rightarrow 0$ for any variable of interest $Z$ defined on $\mathbb{R}^p$. This means that the distance between the sample's measure $\hat{M}$ and the population's measure $M$ goes to zero if and only if the distance between the population total (i.e. $\int Z dM(z)= \sum_{k \in U} z_k$) and the Horvitz-Thompson estimator for the total (i.e. $\int Z d\hat{M}(z)= \sum_{k \in s} z_k/\pi_k$) goes to zero for any variable of interest.

From Assumption 4, then, we have that $\tilde{d}(\hat{M}/N, M/N) = O_p(n^{-1/2})$.

Now because $T$ is Fréchet differentiable by Assumption 7, we can write the following first-order von-Mises expansion (see Huber 1981 pages 35-39 for details):

$$ \begin{aligned}T(\frac{\hat{M}}{N}) &= T(\frac{M}{N}) + \int IT(\frac{M}{N} ; z) d\left(\frac{\hat{M}}{N}-\frac{M}{N}\right)(z) + o\{ \tilde{d}(\hat{M}/N, M/N) \} \\ \end{aligned} $$

Next, we note that for a functional of degree $\beta$, $IT(\frac{M}{N};z)=N^{1-\beta} IT(M,z)$ (proof in Deville, 1999). So we thus have that:

$$ \begin{aligned}T(\frac{\hat{M}}{N}) - T(\frac{M}{N}) &= N^{-\beta} \int IT(M ; z) d\left(\hat{M}-M\right)(z) + o\{ \tilde{d}(\hat{M}/N, M/N) \} \\ \end{aligned} $$

Denote the remainder term $R_n$. Now since we have that $\tilde{d}(\hat{M}/N, M/N)$ is $O_p(n^{-1/2})$, and $R_n$ is $o\left(\tilde{d}(\hat{M}/N, M/N)\right)$, we must have that $R_n=o_p(n^{-1/2})$.

And so we have that:

$$ \begin{aligned} T(\frac{\hat{M}}{N}) - T(\frac{M}{N}) &= N^{-\beta} \int IT(M ; z) d\left(\hat{M}-M\right)(z) + o_p( n^{-1/2} ) \\ \end{aligned} $$

Multiplying both sides by $\sqrt{n}$, we thus have that:

$$ \sqrt{n} \left( T(\frac{\hat{M}}{N}) - T(\frac{M}{N}) \right) = \sqrt{n}N^{-\beta} \int IT(M ; z) d\left(\hat{M}-M\right)(z) + o_p(1) $$

This gives us the desired result:

$$ \begin{aligned} \sqrt{n} N^{-\beta}\left( T(\hat{M}) - T(M) \right) &= \sqrt{n}N^{-\beta} \int IT(M ; z) d\left(\hat{M}-M\right)(z) + o_p(1) \\ &= \sqrt{n}N^{-\beta} \sum_{k=1}^{N} u_k(v_k - 1) + o_p(1) \end{aligned} $$