2
$\begingroup$

Note: This is a follow-up on a previous question that was concerned about consistency, but this time seeking $\sqrt{n}$-consistency.

Suppose we estimate a quantity $\theta_0$ by the $\tilde{\theta} = \hat{\theta}(\eta_0)$ that solves the estimating equation

$$S_n(\tilde{\theta}, \eta_0) = 0$$

where $\eta_0$ is a nuisance parameter that is known. Suppose that the assumptions of the M-estimator are satisfied, with

$$\sqrt{n}(\tilde{\theta}-\theta) = Op(1)$$

so that we have $\sqrt{n}$-consistency.

Question: Let suppose we do not know $\eta_0$, but we have a consistent estimator $\hat{\eta}$ of $\eta_0$. If now $\hat{\theta} = \hat{\theta}(\hat{\eta})$, under which condition do we have $\sqrt{n}$-consistency?

I have already established conditions for simple consistency to hold, but $\sqrt{n}$-consistency seems harder.

$\endgroup$
1
$\begingroup$

Background: Typically, to ensure $\sqrt{n}$-consistency, we assume

1) $\tilde{\theta} \xrightarrow{p}\ \theta_0$

2) $S(\theta,\eta)$ is differentiable in $\theta$ at $(\theta_0,\eta_0)$ with derivative matrix $\Gamma$ of full rank

3) $|S(\tilde{\theta},\eta_0) - S(\theta_0,\eta_0)| = Op(n^{-1/2}) + op(|\tilde{\theta}-\theta|)$

From 1) and 2), we have a $C(\eta_0) > 0$ such that, with probability tending to one,

$$|S(\tilde{\theta},\eta_0) - S(\theta_0,\eta_0)| \ge C(\eta_0) | \tilde{\theta} - \theta_0 |$$

Which means, with 3)

$$\begin{align} |\tilde{\theta} - \theta_0| &= Op(|S(\tilde{\theta},\eta_0) - S(\theta_0,\eta_0)|) \\ &= Op(n^{-1/2})+ op(|\tilde{\theta}-\theta_0|) = Op(n^{-1/2}) \end{align}$$

which proves the result.

Here 3) can be obtained from a variety of assumptions. Typically, we assume that

4) $S_n(\tilde{\theta},\eta_0) = Op(n^{-1/2})$

5) $S_n(\theta_0, \eta_0 ) = Op(n^{-1/2})$

6) $S(\theta_0, \eta_0) = 0$

as well as an additional more technical assumption. One example is

7a) For any sequence $\delta_n$ with $\delta_n \to 0$, $$\sup_{|\theta - \theta_0| < \delta_n} \frac{| S_n(\theta,\eta_0) - S(\theta, \eta_0) - S_n(\theta_0,\eta_0)|}{n^{-1/2} + |\theta - \theta_0| + |S_n(\theta, \eta_0)| + |S(\theta, \eta_0)|} = op(1)$$

Then under 1, 4-6 and 7a we have 3) true.

Proof: Let $\delta_n$ be a sequence that goes to zero such that

$$P( |\tilde{\theta} - \theta_0| > \delta_n ) \to 0$$

Then, we have, with probability tending to one,

$$ \begin{align} |S(\tilde{\theta},\eta_0)| - |S_n(\tilde{\theta},\eta_0)| - |S_n(\theta_0,\eta_0)| &\le |S_n(\tilde{\theta},\eta_0) - S(\tilde{\theta},\eta_0) - S_n(\theta_0,\eta_0)| \\ &= op(n^{-1/2} + |\tilde{\theta} - \theta_0|) \\ &+ op(|S_n(\tilde{\theta},\eta_0)|) + op(|S(\tilde{\theta},\eta_0)|) \end{align}$$

which gives, from 4) and 5),

$$\begin{align} |S(\tilde{\theta},\eta_0)| &= op(n^{-1/2} + |\tilde{\theta} - \theta_0|) + |S_n(\tilde{\theta},\eta_0)|(1 + op(1)) + op(S(\tilde{\theta},\eta_0)) + |S_n(\theta_0,\eta_0)|\\ &= Op(n^{-1/2}) + op(|\tilde{\theta} - \theta_0|) + op(S(\tilde{\theta},\eta_0)) = Op(n^{-1/2}) + op(|\tilde{\theta} - \theta_0|) \end{align}$$

Instead of 7a), we can instead assume

7b)

$$ [S_n(\tilde{\theta},\eta_0) - S(\tilde{\theta},\eta_0)] - [ S_n(\theta_0,\eta_0) - S(\theta_0,\eta_0)] = Op(n^{-1/2}) + op(|\tilde{\theta} - \theta_0|) $$

Straightforward calculus shows that 7b together with 4-6 implies 3.

When $\eta_0$ is unknown, the resulting $\hat{\theta} = \hat{\theta}(\hat{\eta})$ may satisfy 1) (see linked page) and 2) still holds. However, 3) may not hold.

Solution 1:

One way is to assume that in addition to the previous assumption 2), the estimator $\hat{\theta} = \hat{\theta}(\hat{\eta})$ satisfies

A) $\hat{\theta} \xrightarrow{p}\ \theta_0$

B) $|\hat{\eta} - \eta_0| = Op(n^{-1/2}) $

C) $|S(\hat{\theta},\hat{\eta}) - S(\theta_0,\eta_0)| = Op(n^{-1/2}) + op(|\hat{\theta} - \theta_0|)$

D) $S(\theta,\eta)$ is Lipschitz continuous in $\eta$ in a neighborhood of $\theta_0$ and $\eta_0$

Under A-D, condition 3) is satisfied and thus $\hat{\theta}$ has $\sqrt{n}$-consistency.

Proof: From the Lipschitz continuity we have a $K > 0$ such that with probability tending to one

$$| S(\hat{\theta},\eta_0) - S(\hat{\theta},\hat{\eta})| \le K |\hat{\eta} - \eta_0| = Op(n^{-1/2})$$

which implies

$$\begin{align}|S(\hat{\theta},\eta_0) - S(\theta_0,\eta_0)| &\le |S(\hat{\theta},\hat{\eta}) - S(\theta_0,\eta_0)| + |S(\hat{\theta},\hat{\eta}) - S(\hat{\theta},\eta_0)| \\ &= Op(n^{-1/2}) + op(|\hat{\theta} - \theta_0|)\end{align}$$

Solution 2:

Alternatively, if we assume A) and C) together with

2') $S(\theta,\eta)$ is differentiable in $(\theta,\eta)$ at $(\theta_0,\eta_0)$ with derivative matrix $\Gamma$ of full rank

B') $\hat{\eta} \xrightarrow{p}\ \eta_0$

Then we automatically get our result from the derivation in the background.

$\endgroup$
  • $\begingroup$ dont you need some more informations about the speed of $o_p(\widehat{\theta}-\theta_0)$ (which I suppose here stands just for $o_p(\widehat{\theta}-\theta_0)=o_p(1)$?) $\endgroup$ – chRrr Jul 10 '18 at 8:43
  • $\begingroup$ Assumption A) is all you need. Is there a specific equation you have in mind? $\endgroup$ – Guillaume F. Jul 10 '18 at 16:18
  • $\begingroup$ I think my problem is due to your compact listing of the assumptions/setting, i.e. the meaning of $\widehat{\theta}-\theta_0 = O_p(n^{-1/2}) +o_p(\widehat{\theta}-\theta_0)$. Suppose $o_p(\widehat{\theta}-\theta_0) = o_p(n^{-1/4})=O_p(n^{-1/4})$, Then $\sqrt{n}(\widehat{\theta}-\theta)=O_p(1) + O_p(n^{1/4})$ and you would have some problems to derive asymptotic normality. $\endgroup$ – chRrr Jul 11 '18 at 9:37
  • 1
    $\begingroup$ It's one of the properties of $op/Op$ calculus that's never taught explicitly in class but can be deduced from first principles. $$ Z_n = R_n + op(Z_n) \implies Z_n(1+op(1)) = R_n \implies Z_n = (1+op(1))^{-1} R_n \implies Z_n = Op(R_n)$$ $\endgroup$ – Guillaume F. Jul 11 '18 at 14:38
0
$\begingroup$

The other answer doesn't assume that $S_n(\hat{\theta}, \eta_0)$ is differentiable. If we assume $S_n(\hat{\theta}, \eta_0)$ differentiable, our work is simplified somewhat.

Background:

1) $\tilde{\theta} = \theta_0 + op(1)$

2) $S_n(\theta,\eta)$ is equidifferentiable (in probability) in $\theta$ at $(\theta_0,\eta_0)$ with a derivative matrix $\Gamma_n$

3) $\Gamma_n$ in invertible with probability tending to one, with $\Gamma_n^{-1} = Op(1)$

4) $S_n(\tilde{\theta},\eta_0) - S_n(\theta_0,\eta_0) = Op(n^{-1/2})$

With probability tending to one, we can do a Taylor expansion about $\theta_0$,

$$\begin{align} S_n(\tilde{\theta},\eta_0) &= S_n(\theta_0,\eta_0) + \Gamma_n(\tilde{\theta} - \theta_0) + op(\tilde{\theta} - \theta_0) \end{align} $$

Hence

$$\begin{align} \tilde{\theta} - \theta_0 &= Op\left(S_n(\tilde{\theta},\eta_0) - S_n(\theta_0,\eta_0) \right) + op(\tilde{\theta} - \theta_0) = Op(n^{-1/2}) \end{align}$$

Solution:

If we assume A-D,

A) $\tilde{\theta} = \theta_0 + op(1), \hat{\eta} = \eta_0 + op(1) $

B) $S_n(\theta,\eta)$ is uniformly equidifferentiable (in probability) in $\theta$ at $\theta_0$ on a neighborhood $\mathcal{B}$ of $\eta_0$ with a derivative matrix $\Gamma_n(\eta)$

C) On $\mathcal{B}$, $\Gamma_n(\eta)$ is invertible with probability tending to one and $\sup_{\eta \in \mathcal{B}} \Gamma_n^{-1}(\eta) = Op(1)$

D) $S_n(\hat{\theta},\hat{\eta}) - S_n(\theta_0,\hat{\eta}_0) = Op(n^{-1/2}) $

Performing a Taylor expansion about $\theta_0$,

$$\begin{align} S_n(\hat{\theta},\hat{\eta}) &= S_n(\theta_0,\hat{\eta}_0) + \Gamma_n(\hat{\eta})(\hat{\theta} - \theta_0) + op(\hat{\theta} - \theta_0) \end{align} \\ $$

Which implies

$$\begin{align} \hat{\theta} - \theta_0 &= Op\left(S_n(\hat{\theta},\hat{\eta}) - S_n(\theta_0,\hat{\eta}) \right) +op(\hat{\theta} - \theta_0) = Op(n^{-1/2}) \end{align}$$

Which is the result.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.