Questions tagged [asymptotics]

Asymptotic theory studies the properties of estimators and test statistics when the sample size approaches infinity.

Filter by
Sorted by
Tagged with
28
votes
3answers
27k views

Asymptotic distribution of sample variance of non-normal sample

This is a more general treatment of the issue posed by this question. After deriving the asymptotic distribution of the sample variance, we can apply the Delta method to arrive at the corresponding ...
25
votes
2answers
3k views

Why does the continuity correction (say, the normal approximation to the binomial distribution) work?

I wish to better understand how the continuity correction to the binomial distribution for the normal approximation was derived. What method was used to decide we should add 1/2 (why not another ...
25
votes
1answer
2k views

Is there a statistical application that requires strong consistency?

I was wondering if someone knows or if there exists an application in statistics in which strong consistency of an estimator is required instead of weak consistency. That is, strong consistency is ...
25
votes
1answer
2k views

Is there a result that provides the bootstrap is valid if and only if the statistic is smooth?

Throughout we assume our statistic $\theta(\cdot)$ is a function of some data $X_1, \ldots X_n$ which is drawn from the distribution function $F$; the empirical distribution function of our sample is $...
24
votes
2answers
1k views

Why doesn't Wilks' 1938 proof work for misspecified models?

In the famous 1938 paper ("The large-sample distribution of the likelihood ratio for testing composite hypotheses", Annals of Mathematical Statistics, 9:60-62), Samuel Wilks derived the asymptotic ...
22
votes
6answers
10k views

Intuitive understanding of the difference between consistent and asymptotically unbiased [duplicate]

I am trying to to get an intuitive understanding and feel for the difference and practical difference between the term consistent and asymptotically unbiased. I know their mathematical/statistical ...
20
votes
2answers
5k views

Why $\sqrt{n}$ in the definition of asymptotic normality?

A sequence of estimators $U_n$ for a parameter $\theta$ is asymptotically normal if $\sqrt{n}(U_n - \theta) \to N(0,v)$. (source) We then call $v$ the asymptotic variance of $U_n$. If this variance is ...
20
votes
1answer
7k views

Cauchy Distribution and Central Limit Theorem

In order for the CLT to hold we need the distribution we wish to approximate to have mean $\mu$ and finite variance $\sigma^2$. Would it be true to say that for the case of the Cauchy distribution, ...
19
votes
5answers
3k views

When the Central Limit Theorem and the Law of Large Numbers disagree

This is essentially a replication of a question I found over at math.se, which didn't get the answers I hoped for. Let $\{ X_i \}_{i \in \mathbb{N}}$ be a sequence of independent, identically ...
19
votes
3answers
3k views

Asymptotic consistency with non-zero asymptotic variance - what does it represent?

The issue has come up before, but I want to ask a specific question that will attempt to elicit an answer that will clarify (and classify) it: In "Poor Man's Asymptotics", one keeps a clear ...
18
votes
1answer
13k views

What are the regularity conditions for Likelihood Ratio test

Could anyone please tell me what the regularity conditions are for the asymptotic distribution of Likelihood Ratio test? Everywhere I look, it is written 'Under the regularity conditions' or 'under ...
18
votes
2answers
4k views

Observed information matrix is a consistent estimator of the expected information matrix?

I am trying to prove that the observed information matrix evaluated at the weakly consistent maximum likelihood estimator (MLE), is a weakly consistent estimator of the expected information matrix. ...
18
votes
1answer
247 views

Is bootstrap problematic in small samples?

In "3 Things That Bother Me" (1988), Ed Leamer writes: Bootstrap estimates of standard errors are based on the assumption that the observed sample is the same as the true distribution, ...
17
votes
5answers
3k views

Can the empirical Hessian of an M-estimator be indefinite?

Jeffrey Wooldridge in his Econometric Analysis of Cross Section and Panel Data (page 357) says that the empirical Hessian "is not guaranteed to be positive definite, or even positive semidefinite, for ...
16
votes
3answers
921 views

Why doesn't the CLT work for $x \sim poisson(\lambda = 1) $?

So we know that a sum of $n$ poissons with parameter $\lambda$ is itself a poisson with $n\lambda$. So hypothetically, one could take $x \sim poisson(\lambda = 1) $ and say it is actually $\sum_1^n ...
15
votes
5answers
2k views

Approximation error of confidence interval for the mean when $n \geq 30$

Let $\{X_i\}_{i=1}^n$ be a family of i.i.d. random variables taking values in $[0,1]$, having a mean $\mu$ and variance $\sigma^2$. A simple confidence interval for the mean, using $\sigma$ whenever ...
15
votes
1answer
5k views

Large sample asymptotic/theory - Why to care about?

I hope that this question does not get marked "as too general" and hope a discussion gets started that benefits all. In statistics, we spend a lot of time learning large sample theories. We are ...
15
votes
1answer
509 views

Does a Bayesian interpretation exist for REML?

Is a Bayesian interpretation of REML available? To my intuition, REML bears a strong likeness to so-called empirical Bayes estimation procedures, and I wonder if some kind of asymptotic equivalence (...
15
votes
2answers
269 views

Derivation of normalizing transform for GLMs

$\newcommand{\E}{\mathbb{E}}$How is the $A(\cdot) = \displaystyle\int\frac{du}{V^{1/3}(\mu)}$ normalizing transform for the exponential family derived? More specifically: I tried to follow the ...
14
votes
3answers
4k views

Why is the asymptotic relative efficiency of the Wilcoxon test $3/\pi$ compared to Student's t-test for normally distributed data?

It is well-known that the asymptotic relative efficiency (ARE) of the Wilcoxon signed rank test is $\frac{3}{\pi} \approx 0.955$ compared to Student's t-test, if the data are drawn from a normally ...
14
votes
4answers
780 views

Do third order asymptotics exist?

Most asymptotic results in statistics prove that as $n \rightarrow \infty$ an estimator (such as the MLE) converges to a normal distribution based on a second-order taylor expansion of the likelihood ...
13
votes
2answers
4k views

Product and sum of big $O_p$ random variables

I seems that people often use the following properties $O_p(a_n)O_p(b_n) = O_p(a_nb_n)$ and $O_p(a_n)+O_p(b_n) = O_p(a_n+b_n)$. I'm wondering, if these are true for any sequences $a_n,b_n$. The reason ...
13
votes
1answer
11k views

What is the asymptotic covariance matrix?

Is it true that the asymptotic covariance matrix is equal to the covariance matrix of parameter estimates? If not, what is it? And what is the difference between the covariance matrix and the ...
13
votes
1answer
2k views

Is MLE estimation asymptotically normal & efficient even if the model is not true?

Premise: this may be a stupid question. I only know the statements about MLE asymptotic properties, but I never studied the proofs. If I did, maybe I woulnd't be asking these questions, or I maybe I ...
13
votes
2answers
1k views

Hessian of profile likelihood used for standard error estimation

This question is motivated by this one. I looked up two sources and this is what I found. A. van der Vaart, Assymptotic Statistics: It is rarely possible to compute a profile likelihood ...
12
votes
3answers
492 views

Regarding convergence in probability

Let $\{X_n\}_{n\geq 1}$ be a sequence of random variables s.t $X_n \to a$ in probability, where $a>0$ is a fixed constant. I'm trying to show the following: $$\sqrt{X_n} \to \sqrt{a}$$ and $$\frac{...
12
votes
1answer
2k views

Fisher consistency versus "standard" consistency

My question relates two types of consistency. In particular, how does the Fisher consistency differ from standard notions of consistency, such as convergence in probability to the generative parameter....
12
votes
2answers
3k views

How does Pearson's Chi Squared Statistic approximate a Chi Squared Distribution

So if Pearson's Chi Squared Statistic is given for a $1 \times N$ table, then its form is: $$\sum_{i=1}^n\frac{(O_i - E_i)^2}{E_i}$$ Then this approximates $\chi_{n-1}^2$, the Chi-Squared ...
12
votes
1answer
982 views

Approximate distribution of product of N normal i.i.d.? Special case μ≈0

Given $N\geq30$ i.i.d. $X_n\approx\mathcal{N}(\mu_X,\sigma_X^2)$, and $\mu_X \approx 0$, looking for: accurate closed form distribution approximation of $Y_N=\prod\limits_{1}^{N}{X_n}$ asymptotic (...
12
votes
1answer
1k views

When does asymptotic normality of the Bayesian posterior (Bernstein-von Mises) fail?

Consider the posterior density function given (as usual) by $$ \pi(\theta) \prod_{i=1}^n f(x_i;\theta),$$ with prior density $\pi$ and distribution $f(\cdot;\theta)$ of the $n$ observations $x_1, \...
12
votes
1answer
461 views

Asymptotic normality of a quadratic form

Let $\mathbf{x}$ be a random vector drawn from $P$. Consider a sample $\{ \mathbf{x}_i \}_{i=1}^n \stackrel{i.i.d.}{\sim} P$. Define $\bar{\mathbf{x}}_n := \frac{1}{n} \sum_{i=1}^n \mathbf{x}_i$, and $...
12
votes
2answers
613 views

Ratio of sum of Normal to sum of cubes of Normal

Please help me to find the limiting distribution (as $n \rightarrow \infty$) of the following: $$ U_n = \frac{X_1 + X_2 + \ldots + X_n}{X_1^3 + X_2^3 + \ldots X_n^3},$$ where $X_i$ are iid $N(0,1)$.
11
votes
4answers
3k views

How does one explain what an unbiased estimator is to a layperson?

Suppose $\hat{\theta}$ is an unbiased estimator for $\theta$. Then of course, $\mathbb{E}[\hat{\theta} \mid \theta] = \theta$. How does one explain this to a layperson? In the past, what I have said ...
11
votes
2answers
685 views

What happens to the likelihood ratio as more and more data is gathered?

Let $f$, $g$ and $h$ be densities and suppose you have $x_i \sim h$, $i \in \mathbb{N}$. What happens to the likelihood ratio $$ \prod_{i=1}^n \frac{f(x_i)}{g(x_i)} $$ as $n \rightarrow \infty$ ? (...
11
votes
5answers
7k views

Asymptotic distribution of multinomial

I'm looking for the limiting distribution of multinomial distribution over d outcomes. IE, the distribution of the following $$\lim_{n\to \infty} n^{-\frac{1}{2}} \mathbf{X_n}$$ Where $\mathbf{X_n}$ ...
11
votes
2answers
2k views

Latin Hypercube Sampling Asymptotics

I am trying to construct a proof for a problem I am working on and one of the assumptions that I am making is that the set of points I am sampling from is dense over the entire space. Practically, I ...
11
votes
2answers
2k views

Mathematical definition of Infill Asymptotics

I am writing a paper that uses infill asymptotics and one of my reviewers has asked me to please provide a rigorous mathematical definition of what infill asymptotics is (i.e., with math symbols and ...
11
votes
1answer
618 views

Is MLE of $\theta$ asymptotically normal when $(X,Y)\sim e^{-(x/\theta+\theta y)}\mathbf1_{x,y>0}$?

Suppose $(X,Y)$ has the pdf $$f_{\theta}(x,y)=e^{-(x/\theta+\theta y)}\mathbf1_{x>0,y>0}\quad,\,\theta>0$$ Density of the sample $(\mathbf X,\mathbf Y)=(X_i,Y_i)_{1\le i\le n}$ drawn from ...
11
votes
3answers
284 views

How many of the biggest terms in $\sum_{i=1}^N |X_i|$ add up to half the total?

Consider $\sum_{i=1}^N |X_i|$ where $X_1, \ldots, X_N$ are i.i.d. and the CLT holds. How many of the biggest terms add up to half the total sum ? For example, 10 + 9 + 8 $\approx$ (10 + 9 + 8 $\dots$ +...
10
votes
2answers
994 views

Can MCMC iterations after burn in be used for density estimation?

After burn-in, can we directly use the MCMC iterations for density estimation, such as by plotting a histogram, or kernel density estimation? My concern is that the MCMC iterations are not ...
10
votes
1answer
311 views

Density of robots doing random walk in an infinite random geometric graph

Consider an infinite random geometric graph in which the node locations follow a Poisson point process with density $\rho$ and edges are placed between the nodes that are closer than $d$. Therefore, ...
9
votes
2answers
709 views

A continuous function of a sequence of random vectors converges in probability to the function of the limit

Proposition: If $\{ X_n \}$ is a sequence of k-dimensional random vectors s.t. $X_n \overset{p}{\to} X$ and if $g: R^k \rightarrow R^m$ is a continuous mapping, then $g(X_n) \overset{p}{\to} g(X)$. ...
9
votes
1answer
388 views

Conjecture related to Kolmogorov 0-1 Law (for events)

Let $(\Omega, \mathscr F, \mathbb P)$ be a probability space. Conjecture: Suppose we have events $A_1, A_2, ...$ s.t. $\forall \ A \in \bigcap_n \sigma(A_n, A_{n+1}, ...)$, $P(A) = 0$ or $1$. There ...
9
votes
1answer
765 views

Does the definition of regular estimator depend on the rate of convergence? If not, should it?

The definition of regular estimator in my lecture notes is: Let $X_1^{(n)}, \dots, X_n^{(n)} \overset{iid}{\sim} P_n \sim \mathcal{P}(\Theta)$ where $\mathcal{P}(\Theta)$ is a regular parametric ...
9
votes
1answer
269 views

Practical usefulness of pointwise convergence without uniform convergence

Motivation In the context of post-model-selection inference, Leeb & Pötscher (2005) write: Although it has long been known that uniformity (at least locally) w.r.t. the parameters is an important ...
9
votes
1answer
565 views

Example of CLT when moments do not exist

Consider $X_n = \begin{cases} 1 & \text{w.p. } (1 - 2^{-n})/2\\ -1 & \text{w.p. } (1 - 2^{-n})/2\\ 2^k & \text{w.p. } 2^{-k} \text{ for } k > n\\ \end{cases}$ I need to show that ...
9
votes
1answer
567 views

Asymptotic normality of order statistic of heavy tailed distributions

Background: I have a sample which I want to model with a heavy tailed distribution. I have some extreme values, such that the spread of the observations are relatively large. My idea was to model this ...
9
votes
1answer
348 views

Asymptotic distribution of censored samples from $\exp(\lambda)$

Let $X_{(1)}, \ldots, X_{(n)}$ be the order statistic of an i.i.d. sample of size $n$ from $\exp(\lambda)$. Suppose the data is censored so we see only the top $(1-p) \times 100%$ percent of the data, ...
9
votes
1answer
581 views

Simulating Convergence in Probability to a constant

Asymptotic results cannot be proven by computer simulation, because they are statements involving the concept of infinity. But we should be able to obtain a sense that things do indeed march the way ...
9
votes
1answer
2k views

Why don't asymptotically consistent estimators have zero variance at infinity?

I know that the statement in question is wrong because estimators cannot have asymptotic variances that are lower than the Cramer-Rao bound. However, if asymptotic consistence means that an estimator ...

1
2 3 4 5
15