Skip to main content

Questions tagged [asymptotics]

Asymptotic theory studies the properties of estimators and test statistics when the sample size approaches infinity.

Filter by
Sorted by
Tagged with
1 vote
0 answers
26 views

What are the degress of freedom in the summary output for GLMs in R?

I am currently self-studying GLMs with the book "Generalized Additive Models An Introduction with R" and I am a bit confused regarding the degrees of freedom in the summary output for GLMs ...
Dude3400's user avatar
1 vote
0 answers
34 views

Small sample MLE vs OLS efficiency

MLE estimates are asymptotically efficient. Both MLE and OLS estimates are asymptotically normal and for many distributions their limiting variances coincide (information for one observation being the ...
memeplex's user avatar
  • 141
-1 votes
0 answers
36 views

What can we say about the variance of the posterior mean?

In Bayesian inference, there's one famous theorem, Bernstein–von Mises theorem (see the Wikipedia or this lecture notes, page 35), states that in front of sufficiently large samples, that is ...
narip's user avatar
  • 185
0 votes
1 answer
23 views

Unbiased and consistent estimator with positive sampling variance as n approaches infinity? (Aronow & Miller) [duplicate]

In Aronow & Miller, "Foundations of Agnostic Statistics", the authors write on p105: [A]lthough unbiased estimators are not necessarily consistent, any unbiased estimator $\widehat{\...
user24465's user avatar
0 votes
1 answer
85 views

Normal approximation for posterior distribution

I am reading the example 4.3.3 of "The Bayesian Choice" by Christian P. Robert and I was wondering if it is possible to obtain a normal approximation in this case to estimate the posterior. ...
daniel's user avatar
  • 155
3 votes
1 answer
85 views

Unit-Root Asymptotics

I am using the book "Time-series-based econometrics" by Hatanaka to learn about asymptotic theory of unit roots. However, it is quite technical, so I am also using Hamilton's "Time ...
Haus's user avatar
  • 81
2 votes
1 answer
120 views

Sum of asymptotically independent random variables - Convergence

Let $\theta_N=\frac{1}{N}\sum_{i=1}^N \pi_i\cdot g_i$ where $0<\pi_i<1$ and $0<g_i<1/\pi_i$ such that $\theta_N\overset{N\rightarrow \infty}{\rightarrow}\theta$. If $X_i\sim Ber(\pi_i)$, I ...
Pierfrancesco Alaimo Di Loro's user avatar
4 votes
1 answer
269 views

Almost sure convergence using exponential tail bound

I have a question about a theorem in the following set of lecture notes 'A Gentle Introduction to Empirical Process theory' (http://www.stat.columbia.edu/~bodhi/Talks/Emp-Proc-Lecture-Notes.pdf). In ...
Stan's user avatar
  • 385
3 votes
2 answers
170 views

In linear regression, what's the asymptotic distribution of the error variance estimator?

Suppose $$Y_i=X_i'\beta+\epsilon_i$$ with $E(\epsilon_i|X_i)=0$ and $E\epsilon^2_i=\sigma^2$ and I estimate $\sigma^2$ using $s^2=\frac{1}{n}\sum_{i=1}^n (Y_i-X_i'\widehat{\beta})^2$, where $\widehat{...
ExcitedSnail's user avatar
  • 2,966
2 votes
0 answers
20 views

Construct transformations of random variables that are "more normal"

I am reading this page in the Encyclopedia of Mathematics about transformations of random variables. I am puzzled about the Example 2: Let $X_1,...,X_n,...$ be independent random variables, each ...
JJJuuu's user avatar
  • 21
6 votes
1 answer
356 views

Is OLS asymptotically the best estimator even without gaussian error?

It is known that MLE is consistent and asymptotically efficient. OLS under certain assumptions is asymptotically normal. If the errors are gaussian, then OLS is equivalent to MLE. If the errors are ...
user405777's user avatar
0 votes
1 answer
44 views

What is the meaning of $\asymp$ and $\lesssim$ in Martin wainwright's high dim textbook? [closed]

Unfortunately, this text book did not provide a table of notations he used. Can anyone provide me with a definition of $\asymp$ and $\lesssim$ and few examples? For an example in the book, in display (...
Mondayisgood's user avatar
5 votes
2 answers
522 views

Asymptotic unbiasedness + asymptotic zero variance = consistency?

Here, Ben shows that an unbiased estimator $\hat\theta$ of a parameter $\theta$ that has an asymptotic variance of zero converges in probability to $\theta$. That is, $\hat\theta$ is a consistent ...
Dave's user avatar
  • 64.8k
0 votes
1 answer
21 views

Computing the limiting distribution of the Bayes estimator for exponential data with a Gamma prior (by using consistency?)

Let data be $X_i \sim \text{Exp}(\theta)$ iid, $i=1,...,n$. Let the prior be $\text{Gamma}(\alpha, \beta)$. The posterior is then of course $\text{Gamma}(\alpha + n, \beta + \sum X_i)$. The Bayes ...
Featherball's user avatar
0 votes
0 answers
13 views

Asymptotic Distribution and Describe Sources of Increasing Power in an hypothesis testing problem

I am currently dealing with the following problem in a past exam (with no solution): Suppose $S$ follows the Poisson distribution with mean $2\lambda>0$, here $\lambda$ is a parameter. Another two ...
INvisibLE's user avatar
1 vote
1 answer
58 views

Hypothesis testing by asymptotic distribution

Consider the following hypothesis testing problem: under $H_0$: $(X_1,\cdots,X_n) \sim P_n,$ under $H_1$: $(X_1,\cdots,X_n) \sim Q_n.$ We want to show that the minimum testing error goes to zero when $...
efsdfmo12's user avatar
  • 123
5 votes
1 answer
101 views

Is it possible that one estimator performs better than others when sample size $n$ is small but performs worse than others when $n$ is large?

Are there any examples that one estimator performs better than others when sample size $n$ is small but performs worse than others when $n$ is large?
Voyager's user avatar
  • 305
0 votes
0 answers
15 views

Understanding Asymptotic Relative Efficiency and how to compute it

I am learning about asymptotic relative efficiency (ARE) in class, and I am trying to understand exactly how to compute the ARE. From my understanding, asymptotic relative efficiency refers to ...
Harry Lofi's user avatar
2 votes
1 answer
92 views

Non-Parametric Regression with an Omitted Variable

Suppose we use the Kernel Regression Estimator $$\hat{m}(c)=\frac{\sum_{i=1}^n K\left(\frac{x_i-c}{h}\right)y_i}{\sum_{i=1}^n K\left(\frac{x_i-c}{h}\right)}$$ where $h\to 0$ and $nh\to \infty$ as $n\...
Joseph Basford's user avatar
4 votes
1 answer
104 views

Asymptotic results for quantile regression uniformly over all quantiles

In the standard quantile regression (QR) framework, we typically consider only one quantile level of interest, say $\tau$. By standard asymptotic results, we obtain the asymptotic normality of $\sqrt{...
Stan's user avatar
  • 385
0 votes
0 answers
12 views

Engle-Granger cointegration test critical values

I am conducting the Engle-Granger cointegration test on a system of three time series: logged spot exchange rates, logged domestic price index, and logged foreign price index. I would like to use ...
Pavel Filip's user avatar
2 votes
3 answers
81 views

Example of non-consistency of M-estimators in case of pointwise converging criterion functions

When one wants to establish consistency of an M-estimator $\widehat{\theta}_n$, one typically requires uniform convergence of the criterion function $\theta \mapsto M_n(\theta)$. That is, one requires ...
Stan's user avatar
  • 385
0 votes
0 answers
44 views

Bootstrapping t-statistics order of approximation error

Consider bootstrapping the $t$-statistics: $T_n = T(X_1,...,X_n) = \sqrt{n-1}\frac{\bar{X} - \mu}{\hat{\sigma}}$ for iid observations $(X_1,...X_n)$, where $\bar{X} = \frac{1}{n}\sum_{i=1}^{n}X_i$ and ...
fairlife4life's user avatar
5 votes
2 answers
128 views

Sufficient conditions for asymptotic efficiency of MLE

Maximum-likelihood estimators are, according to Wikipedia, asymptotically efficient, that is they achieve the Cramér-Rao bound when sample size tends to infinity. But this seems to require some ...
Luis Mendo's user avatar
  • 1,099
8 votes
2 answers
556 views

Why does the normality assumption not affect Linear Regression in large samples?

I've read once that the normality assumption shouldn't be a problem and that you actually shouldn't care that much if your sample is large. Why is that? Can someone give me a mathematical explanation? ...
dsbr__0's user avatar
  • 827
0 votes
0 answers
17 views

Asymptotic distribution of $U$-statistics

Let $(X_1, Y_1), ...., (X_n, Y_n)$ be iid random vectors with marginal distributions functions $F(x)$ and $G(x)$ (both are continuous distributions) respectively such that $F(0)=G(0)=\frac{1}{2}$. ...
user771946's user avatar
0 votes
2 answers
89 views

Asymptotic standard errors vs exact standard errors

I am getting confused about the derivation of standard errors for the OLS estimator $\widehat{\beta}$. I have seen two different ways to derive standard errors: (i) from the exact covariance matrix of ...
Residual Claimant 's user avatar
1 vote
3 answers
119 views

Asymptotic normality implies consistency

I'm trying without success to solve the following exercise in my econometric textbook: Show that $\sqrt{N}\left(\widehat{\beta_1} - \beta_1 \right) \xrightarrow{d} \mathcal{N}(0,a^2)$, where $a^2$ is ...
Residual Claimant 's user avatar
1 vote
1 answer
89 views

Local Linearity vs Regularity Conditions for the asymptotic distribution of the Likelihood Ratio

In his book 'Asymptotic Statistics,' Aad van der Vaart when discussing the asymptotic distribution of the log-likelihood-ratio says: "The most important conclusion of this chapter is that, under ...
PMTokai's user avatar
  • 11
2 votes
1 answer
81 views

OLS vs WLS in a Heteroskedastic Trending Regression

Suppose we have the following heteroskedastic trending regression model: $$y_i=bi+a_i u_i$$ for some sequence of non-zero constants $a_i$ and $u_i$ an i.i.d. sequence of mean 0 and variance 1 random ...
Joseph Basford's user avatar
0 votes
1 answer
71 views

Asymptotic Normality for GEE Parameters

In the famous Liang and Zeger 1986 paper on GEEs https://www.jstor.org/stable/2336267?seq=9, they sketch a proof using the standard m-estimator arguments: (unstated) regularity conditions + first-...
Winston's user avatar
  • 217
1 vote
1 answer
86 views

Is this a typo on P.75, Theorem 5.52 of the book "Asymptotic Statistics" by Van der Vaart?

Let $\Theta$ be a compact metric space, $\theta \in \Theta.$ Let $m_{\theta}:\mathbb{R}^d\to \mathbb{R}: x\mapsto m_{\theta}(x)$ be a family of measurable function indexed by $\theta \in \Theta.$ Let $...
Mathmath's user avatar
  • 761
1 vote
1 answer
77 views

Asymptotics of $\mathbb E[-\log(p)]$ in a one-sample t-test as $n\to\infty.$

Consider a one-sample two-sided t-test, i.e. $X_1, \ldots, X_n$ are iid. $N(\mu, \sigma)$ random variables and we want to test $H_0\colon \mu=0$ versus $H_A\colon \mu\neq0$. The $t$-statistic is ...
Frederik Ziebell's user avatar
0 votes
0 answers
14 views

References: convergence rates of kernel regression, exchangeable data

I have been studying Kernel estimation; in particular, the Nadaraya-Watson estimator. I am interested in studying the rate of convergence in L^p of the NW (or similar) estimators for subgaussian ...
Rabbithawke's user avatar
3 votes
1 answer
96 views

Prove that the Deviance and the Generalised Pearson Statistic are asymptotically equivalent

I am reading the paper Exponential Dispersion Models from Jørgesen and at page $137$ I have encountered a claim that I don't know how to prove. The author claims that the Generalised Pearson Statistic,...
No-one's user avatar
  • 202
2 votes
1 answer
91 views

Finding the limiting distribution of $\sqrt{n} (\hat{\tau} - \tau)$ as $n \rightarrow \infty$ for $N(\mu, \mu^2 \tau)$

Let $X_i$ for $i = 1, ..., n$ be a random sample from the distribution $N(\mu, \mu^2 \tau)$ with unknown parameters $\mu \in (\infty, 0) \cup (0 ,\infty), \tau > 0$. Find and justify the mle $\hat{\...
Stats_Rock's user avatar
2 votes
2 answers
147 views

Why is the asymptotic bias of the maximum likelihood estimate $b(\theta) = \frac{b_1(\theta)}{n}+\frac{b_2(\theta)}{n^2}+...$?

Firth (1993) states in his introduction that for a $p$-dimensional parameter $\theta$ the asymptotic bias of the maximum likelihood estimate $\hat{\theta}$ may be written as: $b(\theta) = \frac{b_1(\...
Nick Green's user avatar
0 votes
0 answers
79 views

Conditions for existence of KL divergence and unique minimum

Consider two probability density function g(y) and f(y: $\theta$), $\theta \in \Theta$. The KL divergence of f and g is defined by $$ D_{KL}(g|f) := \int \log \frac{g(y)}{f(y: \theta)} \, dy = \...
asdfasdf kansdf's user avatar
0 votes
0 answers
27 views

Asymptotic distribution of $n^{\frac{1}{2}}(\hat{\gamma},\gamma_0)$

I am struggling to understand a proof from Browne M. (1984) Asymptotically distribution-free methods for the analysis of covariance structures. Given $\boldsymbol{\delta_s}=n^{\frac{1}{2}}(\boldsymbol{...
HansKemper's user avatar
0 votes
0 answers
53 views

Standard Error of Truncated Fisher's z Transform

I have a question regarding data following the truncated Fisher's z-transform. I currently understand that the correlation coefficient data, after the unbounded Fisher's z-transform, has a standard ...
Thea Ng's user avatar
2 votes
2 answers
145 views

When stochastic boundedness $O_p(1)$ does not hold

The formal definition of stochastic boundedness $O_p(1)$ of a sequence of random variables $\{X_n\}$ goes $$\{X_n\} = O_p(1) \implies \forall \varepsilon >0, \quad \exists\, N_{\varepsilon}, \...
Alecos Papadopoulos's user avatar
5 votes
0 answers
137 views

Statistical significance tests for neural networks

Horel & Giesecke 2020 developed a statistical signficance test for feature variables in a single-layer feedforward neural network. Namely, fix a probability space $(\Omega, \mathcal{F}, \mathbb{P})...
Sergio Charles's user avatar
0 votes
0 answers
27 views

Estimators that are superefficient on a dense set

In Chapter 8 of van der Vaart's Asymptotic Statistics, it is shown that (under weak regularity conditions) an estimator can be "superefficient" on at most a set of Lebesgue measure zero (...
Potato's user avatar
  • 1,105
1 vote
1 answer
213 views

Is convergence in probability implied by consistency of an estimator?

Every definition of consistency I see mentions something convergence in probability-like in its explanation. From Wikipedia's definition of consistent estimators: having the property that as the ...
Estimate the estimators's user avatar
1 vote
2 answers
53 views

How can we maintain asymptotic normality with slight change?

If $(X_n-\mu_n)/\sigma_n\rightarrow_{d} N(0,1)$ (i.e., $X_n$ is $AN(\mu_n,\sigma_n^2)$), I want to show the following two statements: (1) $X_n$ is $AN(\bar{\mu}_n, \bar{\sigma}_n^2)$ if and only if $\...
Lei's user avatar
  • 11
0 votes
0 answers
20 views

How to know when to use non-parametric coefficient confidence interval estimates for regression?

Say I have either logistic regression or simple linear regression and I am not sure if I have a moderate number of observations, $n = 40$. How do I know when to switch to using a non-parametric ...
Estimate the estimators's user avatar
3 votes
3 answers
302 views

How do we know the distribution of regression coefficients

I'm reading up on asymptotics and hypothesis testing and was thinking about how they link together with regression coefficients. I have read that the CLT shows that the standardised sample mean ...
Geoff's user avatar
  • 741
0 votes
1 answer
148 views

Intuition of Influence Function and Score function: $E[IF(X)S_{\beta}(X; \theta_0)]$

Question I find a theorem regarding influence function and score function \begin{align*} E\left\{IF(Z) S_\beta\left(Z, \theta_0\right)\right\}&=1\\ E\left\{IF(Z) S_\eta^T\left(Z, \theta_0\right)\...
mayu's user avatar
  • 1
2 votes
0 answers
41 views

What is the asymptotic bound for the ratio of sample mean and expectation?

For an i.i.d. observations $X_1,\cdots,X_n$ (bounded), we have the Hoeffding's inequality that establishes the upper bound for the tail probability of $|\bar{X_n}-\mathbb{E}[X_1]|$. I would like to ...
qqhgsjah8221's user avatar
1 vote
2 answers
122 views

Consistency of the pooled standard deviation estimate

Suppose that $X_{ik}\sim\mathcal N(0,\sigma^2)$ for $k = 1,2,\dots, n_i$ are independent and identically distributed for each $i \in\{ 1,2\}$. Note that I assume equal means ($0$) and variances ($\...
Syd Amerikaner's user avatar

1
2 3 4 5
17