35 votes
Accepted

Why does the continuity correction (say, the normal approximation to the binomial distribution) work?

In fact it doesn't always "work" (in the sense of always improving the approximation of the binomial cdf by the normal at any $x$). If the binomial $p$ is 0.5 I think it always helps, except ...
Glen_b's user avatar
  • 283k
26 votes
Accepted

Intuitive understanding of the difference between consistent and asymptotically unbiased

Asymptotic unbiasedness $\impliedby$ consistency + bounded variance Consider an estimator $\hat{\theta}_n$ for a parameter $\theta$. Asymptotic unbiasedness means that the bias of the estimator goes ...
Ben's user avatar
  • 125k
24 votes

Intuitive understanding of the difference between consistent and asymptotically unbiased

They are related ideas, but an asymptotically unbiased estimator doesn't have to be consistent. For example, imagine an i.i.d. sample of size $n$ ($X_1, X_2, ..., X_n$) from some distribution with ...
Glen_b's user avatar
  • 283k
24 votes
Accepted

What's the point of asymptotics?

The first reason we look at the asymptotics of estimators is that we want to check that our estimator is sensible. One aspect of this investigation is that we expect a sensible estimator will ...
Ben's user avatar
  • 125k
20 votes

What kind of distribution is this? $\text{Cov}(X, Y)=0$ but $\text{Corr}(X, Y)=1$

Since the covariance depends on the scale of $X$ and $Y$ and the correlation does not (rescaled to $[-1, -1]$) it is possible. For example, if the variance decreases towards zero: If $X=Y$ and $\...
Pieter's user avatar
  • 2,087
18 votes

What kind of distribution is this? $\text{Cov}(X, Y)=0$ but $\text{Corr}(X, Y)=1$

As far as I can see (perhaps outside of some special circumstances, but you don't mention any), it's not possible. The correlation is the covariance divided by the product of the two standard ...
Glen_b's user avatar
  • 283k
17 votes
Accepted

How does one explain what an unbiased estimator is to a layperson?

Technically what you're describing when you say that your estimator gets closer to the true value as the sample size grows is (as others have mentioned) consistency, or convergence of statistical ...
dsaxton's user avatar
  • 12.1k
17 votes
Accepted

What happens to the likelihood ratio as more and more data is gathered?

If one takes the logarithm of this product, $${\mathfrak{r}}=\log \prod_{i=1}^n \frac{f(x_i)}{g(x_i)} = \sum_{i=1}^n \log\frac{f(x_i)}{g(x_i)}$$and turns it into an average $$\bar{\mathfrak{r}}_n=\...
Xi'an's user avatar
  • 106k
16 votes

Asymptotic distribution of sample variance of non-normal sample

You already have a detailed answer to your question but let me offer another one to go with it. Actually, a shorter proof is possible based on the fact that the distribution of $$S^2 = \frac{1}{n-1} ...
JohnK's user avatar
  • 20.4k
16 votes
Accepted

Limit of $n$ times Beta$(1,n)$ variables when $n$ goes to infinity

First, let's get a sense why this should be true. The density of a Beta$(1,n)$ variable which has been multiplied by $n$ should be proportional to $$\left(\frac{x}{n}\right)^{1-1}\left(1 - \frac{x}{n}...
whuber's user avatar
  • 323k
16 votes
Accepted

When the Central Limit Theorem and the Law of Large Numbers disagree

The error here is likely in the following fact: convergence in distribution implicitly assumes that $F_n(x)$ converges to $F(x)$ at points of continuity of $F(x)$. As the limit distribution is of a ...
Alex R.'s user avatar
  • 13.9k
16 votes

Why does the normality assumption not affect Linear Regression in large samples?

The assumption of the Normality of the error term in a regression that applies Least-Squares estimation methods, is used to make statistical inferences about the coefficients after estimation, it is ...
Alecos Papadopoulos's user avatar
15 votes
Accepted

Is there a result that provides the bootstrap is valid if and only if the statistic is smooth?

$\blacksquare$ (1)Why quantile estimators are not Frechet differentiable but their bootstrap estimator is still consistent? You need Hadamard differentialbility(or compact differentiability depending ...
Henry.L's user avatar
  • 2,470
14 votes
Accepted

How to prove unbiasedness/consistency/normality of an estimator that doesn't have a closed form?

Your estimator is what's known as an M-estimator of $\rho$-type, where in this case $\rho = -f$. If your function $f$ is differentiable, then it is known that under some (fairly strong) conditions, ...
Mathuss's user avatar
  • 466
13 votes

Why doesn't the CLT work for $x \sim poisson(\lambda = 1) $?

I agree with @whuber that the root of the confusion seems to be replacing the summation asymptotic in CLT with some sort of division in your argument. In CLT we get the fixed distribution $f(x,\lambda)...
Aksakal's user avatar
  • 61.3k
13 votes
Accepted

Weak law vs strong law of large numbers - intuition

After a second read of your question, I found that there are two pairs of concepts need clarification. First, the an event happens vs. the probability of an event happens. Second, convergence in ...
Zhanxiong's user avatar
  • 18.8k
12 votes
Accepted

Smarter example of biased but consistent estimator?

Here's a straightforward one. Consider a uniform population with unknown upper bound $$ X \sim U(0, \theta) $$ A simple estimator of $\theta$ is the sample maximum $$ \hat \theta = \max(x_1, x_2, \...
Matthew Drury's user avatar
12 votes

When the Central Limit Theorem and the Law of Large Numbers disagree

For iid random variables $X_i$ with $E[X_i]= \operatorname{var}(X_i)=1$ define \begin{align}Z_n &= \frac{1}{\sqrt{n}}\sum_{i=1}^n X_i,\\ Y_n &= \frac{1}{{n}}\sum_{i=1}^n X_i. \end{align} Now,...
Dilip Sarwate's user avatar
12 votes

Asymptotic Normality and Consistency

I think you may either be confused with the modes of convergence associated with consistency and asymptotic normality or with the definition of consistency. It is important to note that consistency is ...
Ariel's user avatar
  • 2,477
11 votes

Smarter example of biased but consistent estimator?

A very commonly used consistent but biased estimator used is that of the estimated standard deviation. If we are looking at a simple situation in our data is distributed as $x_i \sim N(\mu, \sigma^2)...
Cliff AB's user avatar
  • 21.1k
11 votes

Consistency of lasso

You'll be disappointed to find that the consistency that matters the most with lasso is the consistency about which predictors are chosen. If you simulate two moderately large datasets and perform ...
Frank Harrell's user avatar
11 votes

Generalized CLT for any operation

There can only be such a theorem if $g$ is well-behaved in some sense: in particular, it should only depend on the set of values provided, not the order. Here is a big class of such functions, to ...
chrishmorris's user avatar
  • 1,780
10 votes

What's the convergence rate in the context of convergence in probability?

I would argue that the most widely accepted definition of a convergence rate uses the "big-Oh" and "small-oh" notation. That is, convergence in probability is written as $z_n-z=o_p(1)$, while a rate ...
Christoph Hanck's user avatar
10 votes
Accepted

Asymptotically Normally Distributed

But when we say "an estimator is asymptotically normally distributed", what does it mean? Using similar language to your first sentence, when we say an estimator is asymptotically normally ...
Glen_b's user avatar
  • 283k
10 votes

Does asymptotically unbiased mean convergence in probability?

Let $Y_i \sim \mbox{i.i.d. } N(\mu,1)$. Then $\hat\theta_n=Y_n$ is an unbiased (and hence asymptotically unbiased) estimator of $\mu$. However, it does not converge in probability, and thus is not ...
StasK's user avatar
  • 31.6k
10 votes
Accepted

Does $X\stackrel{d}\to X_1$ and $Y\stackrel{d}\to Y_1$ imply $X+Y\stackrel{d}\to X_1+Y_1$?

What $X+Y$ and $X_1+Y_1$ converge to depends upon the joint distributions of $(X,Y)$ and $(X_1,Y_1)$; what $X, X_1, Y, Y_1$ converge to individually depends upon the marginal distributions. You can ...
jbowman's user avatar
  • 38.7k
10 votes

What is Asymptotic Independence

Assume you have a probability measure $P$. Two events $A$ and $B$ are independent iff $P(A\cap B) = P(A)P(B)$. Two random variables $X$ and $Y$ are independent iff for any measurable sets $A$ and $B$, ...
cgmil's user avatar
  • 1,333
10 votes
Accepted

What are "poor finite sample properties"?

In the context of hypothesis tests, poor finite sample properties usually mean that the actual rejection rate of the test differs from the nominal one. Recall that the nominal one is the level at ...
Christoph Hanck's user avatar
10 votes

Where does e come from in this problem about dice?

The reason $e =\lim\limits_{n \to \infty}\left(1+\frac1n\right)^{n}=\frac1{\lim\limits_{n \to \infty}\left(1-\frac1n\right)^{n}}$ appears is precisely because you are taking a particular limit, though ...
Henry's user avatar
  • 39.6k
9 votes

How does one explain what an unbiased estimator is to a layperson?

I am not sure if you confuse consistency and unbiasedness. Consistency: The larger the sample size the closer the estimate to the true value. Depends on sample size Unbiasedness: The expected value ...
Ferdi's user avatar
  • 5,179

Only top scored, non community-wiki answers of a minimum length are eligible