# Deriving the limiting distribution of the Hodges-Le Cam estimator in Bickel and Doksum (2015)

I am trying to better understand the Hodges-Le Cam estimator, and am having difficulty rendering explicit some of the asymptotic arguments in the derivation of the estimator's limiting distribution. I would appreciate if someone could make certain aspects of the argument explicit by addressing the queries further below.

Context.

The following presentation is extracted from Mathematical Statistics: Basic Ideas and Selected Topics Volume I (2nd ed.) by Bickel and Doksum (2015).

Example 5.4.2. Hodges' Example. Let $$X_1, \dots X_n$$ be i.i.d. $$\mathcal{N}(\theta, 1)$$. Then $$\overline{X}$$ is the MLE of $$\theta$$ and it is trivial to calculate $$I(\theta) \equiv 1$$. Consider the following competitor to $$\overline{X}$$: \begin{align} \tilde{\theta}_n = \begin{cases} 0 \quad \text{if} \quad \vert \overline {X}\vert \leq n^{-1/4} \\ \overline{X} \quad \text{if} \quad \vert \overline {X} \vert > n^{-1/4}. \end{cases} \tag{5.4.37} \end{align} [...]

We next compute the limiting distribution of $$\sqrt{n}(\tilde{\theta} - \theta)$$. Let $$Z \sim \mathcal{N}(0, 1)$$. Then

\begin{align} P_{\theta} [\vert \overline{X} \vert \leq n^{-1/4}] &= P_{\theta}[ \vert Z + \sqrt{n} \theta \vert \leq n^{1/4}] \\ &= \Phi(n^{1/4} - \sqrt{n} \theta) - \Phi(-n^{1/4} - \sqrt{n} \theta). \tag{5.4.38} \end{align}

Therefore if $$\theta \neq 0$$, $$P_{\theta}[\vert \overline{X} \vert \leq n^{-1/4}] \longrightarrow 0$$ because $$n^{1/4} - \sqrt{n} \theta \longrightarrow -\infty$$, and, thus, $$P_{\theta}[\tilde{\theta}_n = \overline{X}] \longrightarrow 1$$. If $$\theta = 0$$, $$P_{\theta}[\vert \overline{X} \vert \leq n^{1/4}] \longrightarrow 1$$, and $$P_{\theta}[\tilde{\theta}_n = 0] \longrightarrow 1$$. Therefore,

\begin{align} \mathcal{L}_{\theta}(\sqrt{n}(\tilde{\theta}_n - \theta)) \longrightarrow \mathcal{N}(0, \sigma^2(\theta)) \\ \tag{5.4.39} \end{align}

where $$\sigma^2(\theta) = 1 = \frac{1}{I(\theta)}$$, $$\theta \neq 0, \sigma^2(0) = 0 < \frac{1}{I(\theta)}$$.

Queries.

1. Where it stated for the case $$\theta \neq 0$$, $$n^{1/4} - \sqrt{n} \theta \longrightarrow - \infty$$, wouldn't this only hold if $$\theta > 0$$? If that is the case, I am unable to understand why the author has omitted dealing with the case where $$\theta < 0$$?

2. What results are being invoked when we go from both $$P_{\theta}[\tilde{\theta} = \overline{X}] \longrightarrow 1$$ for $$\theta \neq 0$$ and $$P_{\theta}[\tilde{\theta}_n = 0] \longrightarrow 1$$ for $$\theta = 0$$ to the two limiting distributions in $$(5.4.39)$$, that is, a standard Normal and a degenerate distribution?

Intuitively, I am partly comfortable with the ideas that:

• As we collect more observations $$n \rightarrow \infty$$, then in the regime where $$\theta \neq 0$$, the Hodges estimator $$\tilde{\theta}_n$$ behaves likes the MLE with increasingly high probability because $$P_{\theta}[\tilde{\theta}_n = \overline{X}] \rightarrow 1$$. And because of the asymptotic normality of MLE, this means that the limiting distribution of $$\sqrt{n}(\tilde{\theta}_n - \theta)$$ is standard normal.

• As we collect more observations $$n \rightarrow \infty$$, then in the regime where $$\theta = 0$$, the Hodges estimator $$\tilde{\theta}_n$$ has increasingly high probability of being set to exactly 0 because $$P_{\theta}[\tilde{\theta} = 0] \rightarrow 1$$. And this means that the limiting distribution of $$\sqrt{n} \tilde{\theta}_n$$ is that of a degenerate CDF at 0.

However, I am having difficulty seeing what results are being used here.

3. Is $$\tilde{\theta}_n$$ a discrete or continuous random variable; or some hybrid that depends on whether $$\theta = 0$$ or $$\theta \neq 0$$?

I am experiencing doubt over this because I am not entirely comfortable with how much sense the statements $$P_{\theta}[\tilde{\theta}_n = \overline{X}]$$ and $$P_{\theta}[\tilde{\theta}_n = 0]$$ make. Because if $$\tilde{\theta}_n$$ is continuous, both probabilities on $$\tilde{\theta}_n$$ taking a value would be 0?

I tried rewriting the Hodges estimator using indicator functions $$\tilde{\theta}_n = \overline{X} \cdot \mathbb{I}(\vert \overline{X} \vert > n^{-1/4})$$, which suggests to me that the intention is that $$\tilde{\theta}_n$$ is continuous when $$\theta \neq 0$$, but I'm not sure about whether $$\tilde{\theta}_n$$ is discrete or continuous when $$\theta = 0$$.

1. Yes, that's for $$\theta>0$$, but the argument for $$\theta<0$$ is exactly symmetric.

2. The key random variable is the indicator $$A_n=\{|\bar X_n|\leq n^{-1/4}\}$$. Since $$\bar X_n\sim N(0,1/n)$$, we know $$P(A_n)$$ exactly. Asymptotically, if $$\theta=0$$, $$P(A_n)\to 1$$. If $$\theta\neq 0$$, $$P(A_n)\to 0$$.

Now, $$\tilde\theta_n= A_n\times 0 + (1-A_n)\times \bar X_n$$ and so $$\sqrt{n}(\tilde\theta-\theta)=A_n\times 0+(1-A_n)\sqrt{n}(\bar X_n-\theta)$$ and by the Continuous Mapping Theorem you get $$\sqrt{n}(\tilde\theta-\theta)\stackrel{p}{\to}0$$ if $$\theta=0$$ and $$\sqrt{n}(\tilde\theta-\theta)\stackrel{p}{\to}N(0,1)$$ if $$\theta=1$$.

So, for any fixed $$\theta$$, the asymptotic distribution is either identical to that of $$\bar X_n$$ (if $$\theta\neq 0$$) or better (if $$\theta=0$$).

1. $$\tilde\theta$$ has a mixed discrete and continuous distribution: it has a point mass at zero, it has zero mass on $$[-n^{-1/4},0)$$ and $$(0,n^{-1/4}]$$, and otherwise it has a strictly positive density.

• $$P(\tilde\theta=0)$$ is well-defined and non-zero; it's just $$P(A_n)$$. It is non-zero because $$\tilde\theta$$ has point mass at zero

• $$P(\tilde\theta=\bar X_n)$$ is well-defined and non-zero; it's just $$1-P(A_n)$$. It is non-zero even though $$\bar X_n$$ is continuous, but that's fine because $$\tilde\theta-\bar X_n$$ has point mass at zero.

2. While you didn't ask, the other interesting part about the Hodges estimator is how it breaks down. If you take $$\theta=n^{-1/4}$$, then $$P(A_n)\approx 1/2$$. When $$A_n=1$$, $$\sqrt{n}(\tilde\theta_n-\theta)\approx n^{1/2}n^{-1/4}=n^{1/4}$$, which is very large for large $$n$$.

• +1. Thank you for the lucid explanation, there were some deficiencies in my understanding (concerning mixed random variables) which I needed to remedy, and for some reason many of the intermediate stats and probability texts say nothing about mixed random variables. I did have other questions which I plan to post separately, I just thought it would be best to secure my understanding of the estimator construction first. On minor notational points, should I read $P(A_n) \rightarrow 1$ as $P(A_n = 1) \rightarrow 1$? And do you mean to write $\overline{X}_n \sim N(0, 1/n)$? May 25, 2021 at 3:07
• As I am not as fluent with asymptotics as I would like, please may you be more literal/explicit with your use of the continuous mapping theorem (CMT)? In particular, I would appreciate if you were to identify the sequence of random variables $S_n$ you are applying the CMT to, the continuous function $g(\cdot)$ to which the result applies; and whether you are using CMT on $S_n \overset{p}{\rightarrow} S$, that is, convergence in probability or on $S_n \overset{d}{\rightarrow} S$, convergence in distribution? Please may you also clarify whether you are using any results other than CMT? May 25, 2021 at 3:08