Asymptotic unbiasedness + asymptotic zero variance = consistency?

Question

Here, Ben shows that an unbiased estimator $\hat\theta$ of a parameter $\theta$ that has an asymptotic variance of zero converges in probability to $\theta$. That is, $\hat\theta$ is a consistent estimator of $\theta$.

It makes sense that we should be able to relax the condition to asymptotic unbiasedness: $\underset{n\rightarrow\infty}{\lim}\mathbb E\left[\hat\theta_n - \theta\right] = 0$. That seems to stay within the spirit of the estimator closing-in on the true parameter value...

...but math has surprised me before.

Can we relax the condition to asymptotic unbiasedness? What is the proof?

Answer 1

Your question may be restated as

Let $\langle\hat{\theta}_n\rangle_{n\in\mathbb N}$ be a sequence of random variables such that \begin{align*} & \theta_n := E[\hat{\theta}_n] \to \theta & \text{ as } n \to \infty. \tag{1}\label{1} \\ & \operatorname{Var}(\hat{\theta}_n) \to 0 & \text{ as } n \to \infty. \tag{2}\label{2} \end{align*} Does $\eqref{1}$ and $\eqref{2}$ imply $\hat{\theta}_n \to_p \theta$?

The answer is in the affirmative and the proof is similar to the original case (i.e., using Chebyshev's inequality, but with slightly more work to split the deviance $|\hat{\theta}_n - \theta|$).

Given $\varepsilon > 0$, by $\eqref{1}$, there exists $N$ sufficiently large such that $|\theta_n - \theta| < \varepsilon/2$ for all $n > N$, it then follows that for all $n > N$, \begin{align*} & P(|\hat{\theta}_n - \theta| > \varepsilon) \\ =& P(|\hat{\theta}_n - \theta_n + \theta_n - \theta| > \varepsilon) \\ \leq & P(|\hat{\theta}_n - \theta_n| > \varepsilon / 2) + P(|\theta_n - \theta| > \varepsilon / 2) \\ =& P(|\hat{\theta}_n - \theta_n| > \varepsilon / 2) \\ \leq & 4\operatorname{Var}(\hat{\theta}_n) / \varepsilon^2. \end{align*} The right-hand side of the inequality converges to $0$ as $n \to \infty$ by $\eqref{2}$, which implies $\hat{\theta}_n \to_p \theta$. This completes the proof.

An equivalent way of seeing it is by decomposing $\hat{\theta}_n - \theta$ as $(\hat{\theta}_n - \theta_n) + (\theta_n - \theta)$ then applying the Slutsky's theorem (or merely the simple asymptotic fact that if $X_n \to_p X$ and $Y_n \to_p Y$ then $X_n + Y_n \to_p X + Y$) -- the first part $\hat{\theta}_n - \theta_n \to_p 0$ is exactly the original case you linked, while the second part $\theta_n - \theta \to 0$ follows by the asymptotic unbiasedness condition $\eqref{1}$.

---

^{At my first reading, I mistakenly interpreted OP's problem as "Does asymptotic unbiasedness alone imply consistency?", which is certainly not true. One counterexample is: let $\theta = 0$, $\hat{\theta}_n$ is a Rademacher random variable, i.e., $P(\hat{\theta}_n = 1) = P(\hat{\theta}_n = -1) = \frac{1}{2}$ for all $n$. It is easy to verify that $E[\hat{\theta}_n] \equiv \theta$ while $\hat{\theta}_n \not\to_p \theta$. Note that in this case $\operatorname{Var}(\hat{\theta}_n) \equiv 1$.}

Answer 2

The following general result would be relevant here:

Given a measure space $(\Omega,\boldsymbol{\mathfrak A},\mu),$ consider a sequence of measurable functions $\langle f_n\rangle_{n\in \mathbb N}$ where $f_n\in L^p(\Omega,\boldsymbol{\mathfrak A},\mu),~p\in[1,\infty).$ If the sequence converges in the $L^p$ norm to $f,$ then $$f_n\overset{\mu}{\to}f.$$

To see that, note for an arbitrary $\varepsilon> 0,$ $$\Vert f_n-f\Vert_p^p\geq \int_{\{\Omega:|f_n-f|\geq \varepsilon\}}|f_n-f|^p~\mathrm d\mu\geq \varepsilon^p\mu\{\Omega:|f_n-f|\geq \varepsilon\},$$ but by assumption $\Vert f_n-f\Vert_p\to 0,$ so $f_n$ converges in measure to $f$ on $\Omega.$

--

Reference:

$\rm[I]$ Real Analysis: Theory of Measure and Integration, J. Yeh, World Scientific, $2014, $ Th. $16.25.$