I have been taught that if an estimator is unbiased, then its convergence in probability can be proven by taking the limit of its variance as the sample size grows to infinity and showing it is equal to 0. However, considering the definition of the variance and the fact the estimator is unbiased, this would also prove mean square convergence, which is a stronger form of convergence. I was therefore wondering if the two concepts are equivalent if the estimator is unbiased, or the method I outlined above is just a sufficient but not necessary condition for convergence in probability of an unbiased estimator. Basically, is it possible to define an unbiased estimator whose variance does not tend to 0, but nonetheless converges in probability?
$\begingroup$
$\endgroup$
3
-
2$\begingroup$ Let $X = X_1, X_2, \ldots, X_n,\ldots$ be a sequence of iid variables with finite expectation. Let $Y = Y_1, \ldots$ be any sequence independent of the first with expectations equal to $0,$ converging in probability to $0,$ but with variances not converging. Consider the estimator $t_n(X) = Y_n + (X_1+\cdots+X_n)/n$ of the common expectation of the $X_i.$ $\endgroup$– whuber ♦Commented Apr 6, 2023 at 19:43
-
1$\begingroup$ Thank you, that's a nice and simple counterexample. $\endgroup$– Francesco BoscoCommented Apr 7, 2023 at 9:28
-
$\begingroup$ @PeterO. Fair enough -- but I considered that comment only a hint, as you can see by comparing it to the answer I have just posted. $\endgroup$– whuber ♦Commented Jul 25, 2023 at 14:06
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
Background and explanation of the question
Recall that an estimator $t$ is a statistic that can be computed from any sufficiently large sample of size $n\ge n_0.$ Thus, if we contemplate collecting a random sample of data and enlarge it indefinitely by obtaining ever more independent data points $X_n$ (thereby modeled as iid random variables), then we obtain a sequence of random variables
$$T_n = t(X_1,X_2,\ldots, X_n),\quad n = n_0, n_0+1, n_0+2,\ldots.$$
$t$ is said to be unbiased when (a) the expectation of $T_n$ is always finite, (b) that expectation does not change with $n,$ and (c) $t$ is considered an "estimator" of this common value $\theta = E[T_n].$ Conditions (a) and (b) imply $\theta$ is a property of the common underlying distribution of the $X_i$ and condition (c) states we intend to use $t$ to guess the value of this property, justifying our use of the term "estimator" and permitting us to refer to $\theta$ as the estimand (or "target") of $t.$
Because each $T_n$ is a random variable it has a distribution function defined by
$$F_n(t) = \Pr(T_n \le t)$$
for all numbers $t.$ The phrase "converges in probability" in the question means that the limiting value of $F_n(t)$ as $n$ grows large is $1$ when $t\gt\theta$ and $0$ when $t\lt\theta$ (which describes the distribution of the random variable whose value is almost surely equal to $\theta$).
A standard method to prove convergence in probability exploits Chebyshev's Inequality. This works provided the variances of the $T_n$ gradually get arbitrarily small: that is, they converge (as a sequence of numbers) to zero.
The question asks whether an unbiased estimator $t$ can converge in probability to its estimand even when the sequence of variances of $T_n$ does not converge to zero (thereby precluding the use of Chebyshev's Inequality).
The answer is yes and the intuition for that is given in a comment I posted to the question. The following solution shows how to convert that intuition into a formal demonstration.
Solution
Consider sampling from a distribution with finite expectation $\theta$ but infinite variance, such as any Student t distribution with degrees of freedom $\nu$ greater than $1$ but $2$ or less that has been shifted by $\theta.$ (That is, when $X$ has such a distribution, we're talking about $\theta + X.$)
Let $t$ be the usual arithmetic mean of the sample. Linearity of expectation implies $t$ is unbiased. The Weak Law of Large Numbers asserts $T_n$ converges to $\theta$ in probability. However, the infinite variance of every $X_n$ implies the variance of $T_n$ is always infinite.
