1
$\begingroup$

Every definition of consistency I see mentions something convergence in probability-like in its explanation.

From Wikipedia's definition of consistent estimators:

having the property that as the number of data points used increases indefinitely, the resulting sequence of estimates converges in probability to $θ_0$.

That sounds like consistency implies convergence in probability. Though, I never see it said so plainly.

Likewise, from this discussion of unbiasedness and consistency:

Roughly speaking, consistency means that for large values of 𝑛 we are going to be close to the true value of the parameter with a high probability

Is it the case that estimator consistency $\iff$ convergence in probability? Consistency seems somewhat informally defined, and I'm not seeing it stated mathematically.

$\endgroup$
2
  • 1
    $\begingroup$ Your first Wikipedia link has a reference to Amemiya, Takeshi (1985) Advanced Econometrics, Definition 3.4.2. That says: "DEFINITION 3.4.2. The estimator $\hat\theta$ of $\theta$ is said to be a consistent estimator if $\textrm{plim } \hat\theta= \theta$. Some authors use the term weakly consistent in the preceding definition, to distinguish it from the term strong consistency used to describe the property $\hat\theta \xrightarrow{\mathrm{a.s.}} \theta$." So Amemiya defines consistency as convergence in probability, while saying others call that weak consistency. $\endgroup$
    – Henry
    Sep 29, 2023 at 22:37
  • $\begingroup$ I see, Thanks. It is odd to me this isn't brought more to the fore in discussions of consistency. It seems so informally defined but actually seems to map to mathematical concepts of convergence. $\endgroup$ Sep 29, 2023 at 22:39

1 Answer 1

2
$\begingroup$

"Consistency $\iff$ convergence in probability" is what I was taught. Formally (definition from Rice's Mathematical Statistics & Data Analysis, Ch. 8.4, 3rd edition):

"Let $\hat{\theta}_n$ be an estimate of a parameter $\theta$ based on a sample of size $n$. Then $\hat{\theta}_n$ is said to be consistent in probability if $\hat{\theta}_n$ converges in probability to θ as $n$ approaches infinity; that is, for any $\epsilon > 0$, $P(|\hat{\theta}_n − \theta| > \epsilon) \to 0$ as $n\to\infty$."

Rice abbreviates "consistent in probability" (from the above definition) to "consistency" for the rest of the section.

My old lecture notes (notes from an older offering here) are even more blunt: a consistent estimator is by definition one for which $\hat{\theta}_n \overset{p}{\to} \theta$.

So I think that, in general, "consistency $\iff$ convergence in probability" is a fair characterization. In any case, the above definitions are the sense in which I hear "consistency" used most often, and maps on to weak consistency as defined by Amemiya & Takeshi (as noted above).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.