Questions tagged [asymptotics]
Asymptotic theory studies the properties of estimators and test statistics when the sample size approaches infinity.
692
questions
1
vote
0answers
32 views
is inverse of 1 + small op(1) equal to 1 + small op(1)?
One rule for small op and big Op is
$$
(1+o_p(1))^{-1} = O_p(1)
$$
(on page 13 of Vaart, A. W. van der. (1998). Asymptotic statistics. Cambridge University Press.)
I am curious whether it is true to ...
1
vote
0answers
31 views
Bias of MLEs increases rather than decreases in n
In the context of conducting simulations to assess the performance of MLE point estimates for truncated data, I am encountering surprising settings in which the bias of MLEs is clearly non-monotonic ...
8
votes
2answers
346 views
MLE and non-normality
What is a non-trivial example of an identifiable model whose MLE is consistent, but the MLE's asymptotic distribution is not normal? Parametric setting and IID sample would be desirable.
2
votes
0answers
22 views
Optimal rate of convergence of nonparametric density estimators
Suppose that $X_1, X_2, \dots, X_n$ forms an independent and identically distributed sample from some $d$-dimensional probability distribution with unknown probability density function $f$. Let $x$ be ...
0
votes
0answers
14 views
Consistent Estimator for the Dispersion of a GLM
I am trying to figure out the proof for consistency of the estimators for an exponential dispersion family. The proof is well covered in the paper "Consistency and Asymptotic Normality of the ...
0
votes
0answers
22 views
"Second order correct", "first order asymptotics"
I keep seeing phrases like "first order asymptotic", "second order correct", "high order asymptotics" and I am honestly don't know how these terms are rigorously/...
2
votes
0answers
15 views
Are there small-sample cases where we can approximate the distribution of the Wald statistic better than with the Normal/Chi-squared distribution?
In my specific case, I'm referring to the parameter estimates of a binomial regression (regardless of whether we are estimating risk ratios or odds ratios). Can the problem of divergence of the Wald ...
0
votes
1answer
30 views
Does $|\hat{\theta}_n|-|\theta|=o_p(\alpha_n)$ implies $\hat{\theta}_n-\theta=o_p(\alpha_n)$
Does $|\hat{\theta}_n|-|\theta|=o_p(\alpha_n)$ implies $\hat{\theta}_n-\theta=o_p(\alpha_n)$? ($\alpha_n\rightarrow 0$ as $n\rightarrow \infty$)
0
votes
1answer
14 views
Rates of convergence with asymptotically negligibly noisy observations
Apologies in advance if this question is not completely well defined. Suppose that I am estimating a nonparametric model for a conditional expectation function $\mathbb E[Y_i | X_i]$ using some i.i.d. ...
1
vote
1answer
33 views
Are two weak instruments better than one weak instrument?
From my understanding, when using IV regression to eliminate confounding effects, we prefer to have a single strong instrument, over multiple weak instruments which can lead to bias.
My question is, ...
0
votes
1answer
99 views
Mixed Model in a repeated measurement design and AUC
My goal is to predict the $Y_i=1$ for each subject $i$ given a set of explanatory variables $x_i$. Since I have repeated measurements for some subjects, I was told to use a mixed model strategy, i.e. ...
2
votes
0answers
35 views
How to calculate the expectation of the KDE using little-o?
This is possibly a duplicate of this question of mine, however, here I ask for clarification regarding an estimation that is done when calculating the expectation of the kernel density estimator (KDE) ...
4
votes
2answers
277 views
Delta method for Poisson ratio
Let $X_1,...,X_n$ be drawn from $Pois(\lambda)$ and $Y_1,...,Y_n$ from $Pois(\theta)$. I would like to find the asymptotic distribution of $$\frac{\overline X}{\overline X + \overline Y }$$ using ...
1
vote
1answer
67 views
Asymptotic Distribution of AUC
It is well known that the area under the curve ($AUC$) is equal to the Mann-Whitney U statistic (c.f. Why is ROC AUC equivalent to the probability that two randomly-selected samples are correctly ...
1
vote
1answer
47 views
Does $X_n=O_p(\alpha_n)$ implies $X_n^{-1}=O_p(1/\alpha_n)$
In which condition can I say that $X_n=O_p(\alpha_n)$ implies $X_n^{-1}=O_p(1/\alpha_n)$, since it holds for $X_n\sim N(0,1)$.(In this case, $X_n=O_p(1)$ and $\frac{1}{X_n}$ is a.s. finite and hence $\...
1
vote
1answer
16 views
What's the rate of $\log|1+\frac{O_p(M^{-1/2})}{fu}|$
What's the rate of $\log|1+\frac{O_p(M^{-1/2})}{fu}|$, where f is a real valued constant, u follows standard normal distribution and hence $u=O_p(1)$. So $\log|1+\frac{O_p(M^{-1/2})}{fu}|=O_p(?)$. I ...
2
votes
0answers
36 views
Parameter estimation in linear regression
This is a regression problem. Let $r_i=f_iu_i+e_i,i=1,...,n$, where $u_i\sim^{i.i.d}N(0,1)$, and $e_i=O_p(\alpha_n),\alpha_n\rightarrow 0$ as $n\rightarrow 0$. When I take the log transformation of $...
2
votes
0answers
49 views
How many samples does one need to perform polynomial regression of degree $m$?
Suppose $(X_i, Y_i)$, $i = 1,\dots, n$ are random variables such that
$$X_i\sim N(0,1)$$
$$Y_i = f(X_i) + \epsilon_i$$
where the $\epsilon_i$ are i.i.d. standard Gaussian and $f(x)=\sum_{k = 0}^\infty ...
2
votes
1answer
87 views
How is the asymptotic justification of the "linearization by influence function method" for surveys established?
The survey R package recently adopted the "linearization by influence function" method of estimating covariances between domain estimates. The central ...
1
vote
1answer
51 views
Fisher vs. Asymptotic Consistency - Example using a single observation as the population mean estimator
I am learning about Fisher Consistency and came across this section of a Wikipedia article (https://en.wikipedia.org/wiki/Fisher_consistency#Relationship_to_asymptotic_consistency_and_unbiasedness) ...
4
votes
1answer
62 views
Asymptotic covariance matrix of $\bar{\pmb x}$
In a text I'm reading it says that we define
$$
\begin{align}
\bar{\pmb x}= \begin{bmatrix}\bar x_1 \\ \bar x_2 \end{bmatrix}
\end{align}
$$
And then immediately says the asymptotic covariance matrix ...
0
votes
0answers
13 views
Estimating rate of convergence from MSE
Given that I have calculated values for the MSE for differing values of $n$ and the estimator $\hat{\theta}$. Is it possible to calculate the rate of convergence, $O_p(n^{-r})$, of this estimator by ...
0
votes
0answers
26 views
Variable converge to known distribution, then what about its likelihood function
If we only know $X$ converge to normal distribution, then is it possible to obtain its likelihood function ? I know it is impossible to get the exact likelihood function.
However, can we claim that ...
0
votes
1answer
91 views
Smoothed CDF to calculate asymptotic normality
If we have the following estimator: $\hat{F_Z}(z)=\frac{1}{N}\sum_{i=1}^N1\{Z_i\leq z\}$. The CDF of $Z$ is defined as $F_Z(z)=Pr(Z\leq z)$. $Z_1, ..., Z_N$ is i.i.d. data.
What would be the steps to ...
0
votes
0answers
20 views
Consistency and Asymptotic Normality in Generalized Linear Mixed Models
I am looking for a general, and perhaps classical, reference with a proof of the consistency and asymptotic normality of marginal maximum likelihood estimators of the parameters of generalized linear ...
0
votes
0answers
23 views
limit behaviour of a quadratic form
I'm reading a book on portfolio optimization and risk management, and I wanna clarify what the author wants to say.
Let $\mathbf{X}=[X_1,...,X_n]$ be a random vector with mean $\mathbf{\mu}=E\{\mathbf{...
6
votes
2answers
388 views
Asymptotic Normality and Consistency
I have difficulties understanding the concept of asymptotic normality and consistency.
Take an estimator of a parameter which is consistent and asymptotically normally distributed. Because it is ...
6
votes
1answer
164 views
Asymptotic bias of LASSO vs. none of SCAD
I am reading a paper which says that LASSO is asymptotically biased while SCAD is not. I take asymptotic (un)biasedness to concern the slope estimators from LASSO and SCAD as the sample size goes to ...
1
vote
0answers
18 views
About the validity of an expansion result
Suppose I have an expansion result: $sup_{p\in P,x\in X}|G_N(p,x)-g_N(p,x)|=O_p(a_N)$. Suppose now I have some (nonparametric) estimator for $p$ denoted by $\widehat{p}=\widehat{p}(x)\in P$ for any $x\...
4
votes
1answer
123 views
Likelihood function as number of observations increases
If we have $n$ iid observations from some $X \sim p(\cdot|\theta)$, what happens to the likelihood function $p(x_1,\dots,x_n|\theta)$ as $n\rightarrow \infty$?
I plotted the product of several $\...
2
votes
1answer
41 views
What, if any, asymptotic arguments are used in moving between various statements of the central limit theorem?
What, if any, asymptotic arguments are used in moving between the various statements of the central limit theorem (e.g. in terms of sample means compared with standardised sample means)?
Context.
My ...
0
votes
1answer
54 views
Reference for regularity conditions for asymptotic of MLE
I wonder if there is a complete list of regularity conditions for MLE asymptotic normality.
I read this post and found a list of 6 conditions but the answer does not include any reference. I read the ...
2
votes
2answers
35 views
Clarification on $m$ and $n$ I'n the $m$-out-of-$n$ bootstrap
I've asked questions on the $m$-out-of-$n$ bootstrap here before. Responses have been quite valuable, but one key aspect still sparks some confusion from non-statisticians.
Chernick (2007, 2011) ...
3
votes
2answers
125 views
Asymptotic distribution of $ \sqrt{n} (θ_n - θ)$ then CI of $θ$
Suppose we have $X_1,...,X_n$ iid with distribution:
$f(x)=xe^{−(\frac{x^2 − θ^2}{2})},x≥θ, θ > 0$
By calculated the median of $f(x)$, $X_{\frac{1}{2}}$ equals $ \sqrt{θ^2 + \log4}$ we obtain an ...
6
votes
1answer
261 views
Basic properties of the kernel density estimator
This is a question from a mathematical statistics textbook, used at the first and most basic mathematical statistics course for undergraduate students. This exercise follows the chapter on ...
3
votes
1answer
45 views
Delta Method around zero is a N(0, 0)
I have this problem: $\sqrt N \hat{\theta} \sim N(0, V)$ where $E(\hat{\theta}) = \theta_{0} = 0$. I must find the asymthotic distribution of $\frac{N}{V}\hat{\theta}^{2}$ but if I use the Delta ...
4
votes
1answer
103 views
Asymptotic distribution of OLS standard errors
Consider the linear regression model:
$y_i = \beta + u_i$. We might write this as:
$$Y = X\beta + U\text{ with }Y = \begin{pmatrix}y_1 \\
y_2 \\
\vdots \\
y_n\end{pmatrix}, X = \begin{pmatrix}1 \\
1 \\...
0
votes
0answers
24 views
Standard Error of ERGM Coefficients
I am trying to calculate the standard error of ERGM coefficients, which is estimated by MCMC sample.
For an ERGM $P(y;\eta) = \exp[\eta^\top g(y) - \psi(\eta)]$, denote $\eta$ as the true parameter, $\...
4
votes
1answer
144 views
Deriving the limiting distribution of the Hodges-Le Cam estimator in Bickel and Doksum (2015)
I am trying to better understand the Hodges-Le Cam estimator, and am having difficulty rendering explicit some of the asymptotic arguments in the derivation of the estimator's limiting distribution. I ...
3
votes
1answer
174 views
Asymptotic normality of MLE
We know under regularity conditions the MLE is asymptotically normal. Usually, it is said that in practice it's hard to check these assumptions. However, I wondered whether we can claim that these ...
2
votes
0answers
32 views
Asymptotic Variances of MLE with Orthogonality Constraints
I have a model that is parameterized by orthogonal matrix $\boldsymbol \Gamma$ and other unconstrained parameters $\theta$ that are closely related to $\boldsymbol \Gamma$ within the model. ...
4
votes
1answer
87 views
Asymptotic chi-squared distribution of likelihood ratio statistic in regression problem
There is a famous result, going back to Wilks (1938) "The large-sample distribution of the likelihood ratio for testing composite hypotheses" (Ann. Math. Stat., 9, 60-62) that states that ...
1
vote
0answers
38 views
Asymptotic efficiency of estimators of autoregressive models
Are OLS or MLE estimators of autoregressive model asymptotically efficient if errors are i.i.d?
Consider the case of an AR(1) model $$x_t=\alpha x_{t-1} + \epsilon_t$$
with $\epsilon_t$ ~ $i.i.d. N(0,\...
1
vote
1answer
30 views
Normal approximation and Hoeffding bound
Hoeffding bound for any $\epsilon>0$ is:
$$P_F(|\bar{X}_n-\mu(F)|\geq \epsilon)\leq 2 \exp\{-\frac{n\epsilon^2}{2}\}=h(\sqrt{n}\epsilon)$$ wherever $|X|<1$.
Now I want to have a comparison ...
5
votes
1answer
69 views
Convergence of uniformely distributed random variables on a sphere
I am reading "Asymptotic Statistics" by A.W van der Vaart and I am stuck with an exercise of chapter 2.
Here is the question : for each $n \in \mathbb{N}$, let $U_n$ be uniformly distributed ...
1
vote
1answer
50 views
Why approximate delta-method Variance isn't multiplied by $\frac{1}{n}$?
I'm reading Casella-Berger chapter 10, where they introduce asymptotic evaluations.
I don't seem quite to understand how the factor $\sqrt{n}$ works when we are using asymptotic evaluations in order ...
0
votes
0answers
23 views
Local Data Generating Process in Semiparametric Statistics
I am a bit confused about the LDGP assumption that is mentioned in books on semiparametric statistics. For example, in Semiparametric Theory and Missing Data by Tsiatis, the LDGP is defined as follows:...
0
votes
1answer
18 views
OLS Consistency for Count Data?
Why would we choose Poisson / NB regression (GLM) over OLS for fitting count data?
Is there a way to show that OLS estimator would lost it consistency and asymptotic normality for count data?
I'm ...
0
votes
0answers
44 views
relation between wald statistic and f statistic
$Y=X_i\beta_i+X_2\beta_2+\epsilon$
Y and $\epsilon$ are n*1 matrix
$X_1$ is $n*k_1$ matrix
$X_2$ is $n*k_2$ matrix
$\beta_1$ is $k_1*1$ matrix
$\beta_2$ is $k_2*1$ matrix
F is f statistic for testing $...
1
vote
0answers
28 views
Different regularity conditions for finite population CLT
I am having trouble understanding the different regularity conditions for different versions of the finite population central limit theorem. I would greatly appreciate any help or insight anyone has.
...
