4
$\begingroup$

I am trying to better understand the Hodges-Le Cam estimator, and am having difficulty rendering explicit some of the asymptotic arguments in the derivation of the estimator's limiting distribution. I would appreciate if someone could make certain aspects of the argument explicit by addressing the queries further below.

Context.

The following presentation is extracted from Mathematical Statistics: Basic Ideas and Selected Topics Volume I (2nd ed.) by Bickel and Doksum (2015).

Example 5.4.2. Hodges' Example. Let $X_1, \dots X_n$ be i.i.d. $\mathcal{N}(\theta, 1)$. Then $\overline{X}$ is the MLE of $\theta$ and it is trivial to calculate $I(\theta) \equiv 1$. Consider the following competitor to $\overline{X}$: \begin{align} \tilde{\theta}_n = \begin{cases} 0 \quad \text{if} \quad \vert \overline {X}\vert \leq n^{-1/4} \\ \overline{X} \quad \text{if} \quad \vert \overline {X} \vert > n^{-1/4}. \end{cases} \tag{5.4.37} \end{align} [...]

We next compute the limiting distribution of $\sqrt{n}(\tilde{\theta} - \theta)$. Let $Z \sim \mathcal{N}(0, 1)$. Then

\begin{align} P_{\theta} [\vert \overline{X} \vert \leq n^{-1/4}] &= P_{\theta}[ \vert Z + \sqrt{n} \theta \vert \leq n^{1/4}] \\ &= \Phi(n^{1/4} - \sqrt{n} \theta) - \Phi(-n^{1/4} - \sqrt{n} \theta). \tag{5.4.38} \end{align}

Therefore if $\theta \neq 0$, $P_{\theta}[\vert \overline{X} \vert \leq n^{-1/4}] \longrightarrow 0$ because $n^{1/4} - \sqrt{n} \theta \longrightarrow -\infty$, and, thus, $P_{\theta}[\tilde{\theta}_n = \overline{X}] \longrightarrow 1$. If $\theta = 0$, $P_{\theta}[\vert \overline{X} \vert \leq n^{1/4}] \longrightarrow 1$, and $P_{\theta}[\tilde{\theta}_n = 0] \longrightarrow 1$. Therefore,

\begin{align} \mathcal{L}_{\theta}(\sqrt{n}(\tilde{\theta}_n - \theta)) \longrightarrow \mathcal{N}(0, \sigma^2(\theta)) \\ \tag{5.4.39} \end{align}

where $\sigma^2(\theta) = 1 = \frac{1}{I(\theta)}$, $\theta \neq 0, \sigma^2(0) = 0 < \frac{1}{I(\theta)}$.

Queries.

1. Where it stated for the case $\theta \neq 0$, $n^{1/4} - \sqrt{n} \theta \longrightarrow - \infty$, wouldn't this only hold if $\theta > 0$? If that is the case, I am unable to understand why the author has omitted dealing with the case where $\theta < 0$?

2. What results are being invoked when we go from both $P_{\theta}[\tilde{\theta} = \overline{X}] \longrightarrow 1$ for $\theta \neq 0$ and $P_{\theta}[\tilde{\theta}_n = 0] \longrightarrow 1$ for $\theta = 0$ to the two limiting distributions in $(5.4.39)$, that is, a standard Normal and a degenerate distribution?

Intuitively, I am partly comfortable with the ideas that:

  • As we collect more observations $n \rightarrow \infty$, then in the regime where $\theta \neq 0$, the Hodges estimator $\tilde{\theta}_n$ behaves likes the MLE with increasingly high probability because $P_{\theta}[\tilde{\theta}_n = \overline{X}] \rightarrow 1$. And because of the asymptotic normality of MLE, this means that the limiting distribution of $\sqrt{n}(\tilde{\theta}_n - \theta)$ is standard normal.

  • As we collect more observations $n \rightarrow \infty$, then in the regime where $\theta = 0$, the Hodges estimator $\tilde{\theta}_n$ has increasingly high probability of being set to exactly 0 because $P_{\theta}[\tilde{\theta} = 0] \rightarrow 1$. And this means that the limiting distribution of $\sqrt{n} \tilde{\theta}_n$ is that of a degenerate CDF at 0.

However, I am having difficulty seeing what results are being used here.

3. Is $\tilde{\theta}_n$ a discrete or continuous random variable; or some hybrid that depends on whether $\theta = 0$ or $\theta \neq 0$?

I am experiencing doubt over this because I am not entirely comfortable with how much sense the statements $P_{\theta}[\tilde{\theta}_n = \overline{X}]$ and $P_{\theta}[\tilde{\theta}_n = 0]$ make. Because if $\tilde{\theta}_n$ is continuous, both probabilities on $\tilde{\theta}_n$ taking a value would be 0?

I tried rewriting the Hodges estimator using indicator functions $\tilde{\theta}_n = \overline{X} \cdot \mathbb{I}(\vert \overline{X} \vert > n^{-1/4})$, which suggests to me that the intention is that $\tilde{\theta}_n$ is continuous when $\theta \neq 0$, but I'm not sure about whether $\tilde{\theta}_n$ is discrete or continuous when $\theta = 0$.

$\endgroup$
4
$\begingroup$
  1. Yes, that's for $\theta>0$, but the argument for $\theta<0$ is exactly symmetric.

  2. The key random variable is the indicator $A_n=\{|\bar X_n|\leq n^{-1/4}\}$. Since $\bar X_n\sim N(0,1/n)$, we know $P(A_n)$ exactly. Asymptotically, if $\theta=0$, $P(A_n)\to 1$. If $\theta\neq 0$, $P(A_n)\to 0$.

Now, $$\tilde\theta_n= A_n\times 0 + (1-A_n)\times \bar X_n$$ and so $$\sqrt{n}(\tilde\theta-\theta)=A_n\times 0+(1-A_n)\sqrt{n}(\bar X_n-\theta)$$ and by the Continuous Mapping Theorem you get $$\sqrt{n}(\tilde\theta-\theta)\stackrel{p}{\to}0$$ if $\theta=0$ and $$\sqrt{n}(\tilde\theta-\theta)\stackrel{p}{\to}N(0,1)$$ if $\theta=1$.

So, for any fixed $\theta$, the asymptotic distribution is either identical to that of $\bar X_n$ (if $\theta\neq 0$) or better (if $\theta=0$).

  1. $\tilde\theta$ has a mixed discrete and continuous distribution: it has a point mass at zero, it has zero mass on $[-n^{-1/4},0)$ and $(0,n^{-1/4}]$, and otherwise it has a strictly positive density.

    • $P(\tilde\theta=0)$ is well-defined and non-zero; it's just $P(A_n)$. It is non-zero because $\tilde\theta$ has point mass at zero

    • $P(\tilde\theta=\bar X_n)$ is well-defined and non-zero; it's just $1-P(A_n)$. It is non-zero even though $\bar X_n$ is continuous, but that's fine because $\tilde\theta-\bar X_n$ has point mass at zero.

  2. While you didn't ask, the other interesting part about the Hodges estimator is how it breaks down. If you take $\theta=n^{-1/4}$, then $P(A_n)\approx 1/2$. When $A_n=1$, $\sqrt{n}(\tilde\theta_n-\theta)\approx n^{1/2}n^{-1/4}=n^{1/4}$, which is very large for large $n$.

$\endgroup$
2
  • $\begingroup$ +1. Thank you for the lucid explanation, there were some deficiencies in my understanding (concerning mixed random variables) which I needed to remedy, and for some reason many of the intermediate stats and probability texts say nothing about mixed random variables. I did have other questions which I plan to post separately, I just thought it would be best to secure my understanding of the estimator construction first. On minor notational points, should I read $P(A_n) \rightarrow 1$ as $P(A_n = 1) \rightarrow 1$? And do you mean to write $\overline{X}_n \sim N(0, 1/n)$? $\endgroup$
    – microhaus
    May 25 at 3:07
  • $\begingroup$ As I am not as fluent with asymptotics as I would like, please may you be more literal/explicit with your use of the continuous mapping theorem (CMT)? In particular, I would appreciate if you were to identify the sequence of random variables $S_n$ you are applying the CMT to, the continuous function $g(\cdot)$ to which the result applies; and whether you are using CMT on $S_n \overset{p}{\rightarrow} S$, that is, convergence in probability or on $S_n \overset{d}{\rightarrow} S$, convergence in distribution? Please may you also clarify whether you are using any results other than CMT? $\endgroup$
    – microhaus
    May 25 at 3:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.