9
$\begingroup$

I have recently finished studying the central limit theorem and the idea of consistency. I am still a little fuzzy about them, so I was wondering what are some key similarities and differences of the two. Can they be used together?

$\endgroup$
1
  • 6
    $\begingroup$ It is the law of large numbers that is more directly related to consistency (at least for sample averages converging to expected values) than the CLT is. $\endgroup$ – Richard Hardy Apr 1 at 18:50
16
$\begingroup$

Consistency is a property of an estimator.

The central limit theorem is, well, a theorem: it relates to the asymptotic property of the sample average under certain conditions, and that they tend to a normal distribution with variance equal to the inverse of the information matrix at a rate of root-n.

Not all estimators are sample averages. And if they're lucky enough to have an asymptotic distributions, it may not be normal.

For instance, you can estimate the upper bound of a uniform (0, $\theta$) distribution by $X_{(n)}$, the sample maximum. This is a biased estimator, but it is consistent because the bias goes to 0 in large samples. The bias can be corrected by a factor of $n/(n-1)$ and this estimator has an asymptotically exponential distribution due to Huzurbazar.

$\endgroup$
1
$\begingroup$

Here is my attempt to explain the similarities and differences between the CLT and consistency from a statistical point of view using a particular example. My main focus here is intuition and I completely understand that this example is not flawless but I hope this is still useful.

Suppose that $X_1,\ldots,X_n$ are iid random variables such that $\operatorname EX_1=\mu$ and $\operatorname{Var}X_1=1$. Our goal is to learn the value of $\mu$ (which is unobservable) using the observations $X_1,\ldots,X_n$.

The natural estimator of $\mu$ is the sample mean given by $$ \bar X_n=\frac1n\sum_{k=1}^nX_k $$ for $n\ge1$. Of course, we need to investigate how good this estimator is. For example, are we be able to obtain more and more accurate information when the sample size increases? The answer is yes and that is what consistency tells us. For all $\varepsilon>0$, we have that $$ P(|\bar X_n-\mu|>\varepsilon)\to0 $$ as $n\to\infty$. In simple terms, if we choose any accuracy level $\varepsilon>0$, the probability that the value of the estimator is further away from $\mu$ than the accuracy level $\varepsilon>0$ gets smaller and smaller when the sample size increases. However, we are not able to saying anything beyond that.

If we want to know more, we can investigate the error that we make by estimating $\mu$ further. Using the CLT, we have that $$ \sqrt n(\bar X_n-\mu)\to N(0,1) $$ as $n\to\infty$. Observe that the CLT implies consistency (see here). This tells us that not only the error goes to $0$ in probability as $n\to\infty$ but also gives us some approximate information about the distribution of the error. For large values of $n$, $$ P(|\bar X_n-\mu|> z_{\alpha/2}/\sqrt n)\approx\alpha, $$ where $z_{\alpha/2}$ is the $\alpha/2$-level quantile of the standard normal distribution.

Intuitively, we can think that we are interested in the same thing: the accuracy of $\bar X_n$. This is what is similar between consistency and the CLT. The difference between consistency and the CLT is the information that we obtain. If we only have consistency, we know that the probability to observe a large error gets smaller and smaller when the sample size increases. If we have the CLT, in addition to having consistency, we are able to obtain approximate information about the distribution of the error.

I hope this is useful.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.