Questions tagged [asymptotics]

Asymptotic theory studies the properties of estimators and test statistics when the sample size approaches infinity.

0
votes
0answers
10 views

Hausman test statistic - to multiply or not to multiply by n

I am having some serious doubts regarding the formula of the Hausman statistic for the case in which I compare OLS and IV estimates. I am getting confused with what my references are giving me. What ...
0
votes
0answers
33 views

Limiting Distribution of $n\left[Y_n\right]$ where $Y_n$ is the minimum of a sample of size n from Uniform$\left(0,\theta\right)$ distribution

Suppose $X_1,X_2,\dots,X_n$ is a random sample from Uniform$(0,\theta)$ for some unknown $\theta > 0$. Let $Y_n$ be the minimum of $X_1,X_2,\dots,X_n$. (a) Suppose $F_n$ is the CDF of $nY_n$. Show ...
0
votes
1answer
14 views

Why are oracle inequalities called that way?

The oracle property is an asymptotic property of an estimator, and is about variable selection: An estimator $\hat \beta_n$ satisfies the oracle property if in the limit of $n\to \infty$, the ...
2
votes
0answers
31 views

Asymptotic normality of quadratic form?

Let $X$ be a $p$-dimensional vector that is asymptotically normal such that $$\sqrt{n}(X - \mu_X) \stackrel{d}\longrightarrow N(0, \Sigma)$$, and let $H$ be a random $p\times p$ symmetric matrix, ...
1
vote
0answers
39 views

Distribution of variance of $X(X^TX)^{-1}X^TZ$?

I had a question that I couldn't quite wrap my mind around. Essentially, it's this. Suppose $x_i$ is a random $k$-vector, and $X$ is an $n\times k$ vector that is $n$ i.i.d. copies of this $x_i$ "...
1
vote
0answers
17 views

Counting the number of tuples $(i,i',k,k')$ satisfying some conditions

I'm struggling to find the cardinality of a set that will be presented below, as well as an upper bound for it. It is an engaging problem and perhaps not trivial. I will give now the baseline of the ...
0
votes
0answers
24 views

Generalizing Bayesian methods by assuming a “distribution of distributions” instead of a prior

Bayesian methods assume a prior distribution with several hyperparameters. Unfortunately, this is asymptotically incorrect, because distributions in the real world are never exact. For example, the ...
0
votes
0answers
25 views

Finding a test using asymptotic theory. for Poisson $(\lambda)$

If we have a sample of Poisson $(\lambda)$ (a) Find a test for $H_0: \lambda =2$ vs $ H_a: \lambda =\lambda_1> 2$ (b) Find a test using asymptotic theory. (c) Compare the results in en (a) y (...
2
votes
0answers
62 views

Asymptotic relationship between tests of compact composite nulls and one sided tests

A motivating example Let $x_t\sim N(\mu,1)$ for $t=1,\dots,T$. Consider testing the null $H_0: \mu\in[-1,1]$ against the alternative $H_1:\mu\in\mathbb{R}\setminus [-1,1]$. The natural (UMPU?) test ...
2
votes
0answers
27 views

Understanding simple LRT test asymptotic using Taylor expansion?

I am trying to understand the proof that the LRT test for $$H_0: \theta = \theta_0 \quad vs \quad H_1: \theta \neq \theta_0$$ is asymptotically chisquared distributed with one degree of freedom. I ...
0
votes
0answers
14 views

confidence interval for the variance with unknown distribution

Let $X_i \sim \text{iid}(\mu, \sigma^2)$ (the question does not specify whether or not $\mu$ and $\sigma^2$ are known). I have to show the confidence interval for the variance. Since $X_i$ is not ...
0
votes
0answers
12 views

When is the bias of a statistic of the form a/n + b/n^2 + c/n^3 +

In many books the bias-correction of the Jackknife resampling method is being prooved under the assumption, that the bias has a special form, namely a/n + b/n^2 + c/n^3 * ... Sometimes it's written "...
6
votes
2answers
109 views

What's an example of an estimator that is $o_p(n^{-1/2})$?

For $X_i$ i.i.d. normal with mean $\mu < \infty$ and variance 1, by the law of large numbers, we have $\bar{X} \overset{\mathcal{P}}{\rightarrow}\mu$, i.e. $\bar{X} - \mu = o_p(1)$. Similarly, by ...
1
vote
0answers
30 views

Approximating the Kullback-Leibler Divergence with a Laplace approximation

Suppose I wish to compute the (asymptotic) Kullback-Leibler Divergence (KLD) between the exact Bayesian posterior $$q_{n}(\theta|x_{1:n}) \propto \pi(\theta)\prod_{i=1}^n p(x_i|\theta)$$ and the ...
0
votes
0answers
23 views

Bound of the bias of a Gaussian Process by its standard deviation in Gaussian Process Regression

In Gaussian Process Regression (GPR), intuitively, the bias of the conditioned Gaussian Process (posterior) at a location $x^*$ gets smaller if the variance at $x^*$ is getting smaller, for example in ...
9
votes
1answer
173 views

Is MLE of $\theta$ asymptotically normal when $(X,Y)\sim e^{-(x/\theta+\theta y)}\mathbf1_{x,y>0}$?

Suppose $(X,Y)$ has the pdf $$f_{\theta}(x,y)=e^{-(x/\theta+\theta y)}\mathbf1_{x>0,y>0}\quad,\,\theta>0$$ Density of the sample $(\mathbf X,\mathbf Y)=(X_i,Y_i)_{1\le i\le n}$ drawn from ...
0
votes
1answer
44 views

Approximate distribution for sum of squares of standardized Poisson random variables

Suppose that $X_1, ..., X_n$ are independent and identically distributed Poisson($\lambda$) random variables. What is a good approximating distribution for $\sum_{i = 1}^{200} \frac{(X_i - \lambda)^2}...
1
vote
1answer
33 views

Convergence of error term

I was deriving the ratio of a Laplace approximation with the true quantity and I got this: $$ \left(n\bar{x} + \alpha - \frac{1}{2}\right)\log\left(\frac{\bar{x}+(a-1)/n}{\bar{x}+\alpha/n}\right) + n\...
0
votes
0answers
15 views

The Cochran and Cox Approximation Pair T test Unequal Variance

Hi everyone I hope you are well. Maybe as you know according to Behners-Fisher problem (unequal variance case of samples) there are some kind of approximations. I have recently covered the ...
0
votes
0answers
25 views

Convergence in probability (asymptotic notation) result

Let $h=h_n$ be a sequence of numbers such that $h_n \rightarrow 0$ as $n \rightarrow \infty$, $\mu$ be a real constant and $f$ be some probability density function. I was wondering if the following ...
0
votes
0answers
18 views

The limit distribution of Wilcoxon signed rank statistic?

An alternative representation of the Wilcoxon signed rank statistic $V$ is $V=\sum_{i\le j}\mathbb{I}_{\{X_i+X_j>0\}}=\sum_i\mathbb{I}_{\{X_i>0\}}+\sum_{i<j}\mathbb{I}_{\{X_i+X_j>0\}}$ ...
3
votes
1answer
68 views

AIC for increasing sample size

I am using AIC as a model selection criteria in one of my projects. However, since AIC isn't dependent on the number of points sampled, for large n the log likelihood term rapidly outscales the ...
3
votes
2answers
132 views

What “nice” property of a confidence set is being violated

Suppose that I have $X_{i} \overset{i.i.d.}{\sim} P$ with $E[X_{i}]=\mu$ and $V[X_{i}^{2}] = \sigma^{2}<\infty$. Then by the central limit theorem I know that: \begin{align} \sqrt{n} (\bar{X}_{n} ...
0
votes
0answers
19 views

Behavior of likelihood far from peak

In the neighborhood of the maximum likelihood point the log-likelihood function is often fruitfully expanded as a quadratic function of the parameters. Are there any general results about the shape ...
2
votes
1answer
128 views

How to derive the asymptotic distribution of t-statistic?

Let ${X_n}$ be an IID sample such that ${X_i} \sim N(\mu,\sigma^2)$. When both $\mu$ and $\sigma$ are unknown, we construct $t(\hat{\mu},s)=\dfrac{\sqrt{n}(\hat{\mu}-\mu)}{s}$, where $s$ is the sample ...
4
votes
1answer
70 views

Convergence of scaled $L_1$ distance between two sorted random vectors with same limiting distribution

Let $ X=(X_1,\dots,X_n) $ and $ Y=(Y_1,\dots,Y_n) $ where the RVs $ X_1,\dots X_n, Y_1,\dots Y_n $ are independent and have the same limiting distribution (assume for simplicity that all moments exist)...
1
vote
0answers
20 views

Type 2 error in t-test on time series

I have an AR(1) time series with $1>\phi>0$. If I naively use t-test to check $H_0:\mu=0$ and it does not reject the null, then can I accept the result? I think yes because for a time-series ...
0
votes
0answers
18 views

Asymptotic probability that chi-squared distribution exceeds its mean

Let $\chi_k$ be the chi-squared distribution with $k$ degrees of freedom. $\chi_k$ has mean $k$. Is it true that $$\lim_{k \to \infty} \Pr[\chi_k \geq k] = 1/2$$ This seems pretty plausible.
0
votes
0answers
27 views

Consistency of variance estimator in OLS [duplicate]

Given the model, $$ y_i = x_i'\beta + \epsilon_i \quad \epsilon_i \sim N(0, \sigma^2) \quad iid \quad \forall i = 1, ..,n $$ how can I prove that the estimator of the variance $\hat{\sigma}^2 = \...
0
votes
0answers
33 views

Tail behaviour of normal cdf?

Q: What is the tail behaviour of $\log \Phi(t)$ as $t \to \infty$? Since $\Phi(t) \to 1$ as $t \to \infty$, we know that $\log \Phi(t)\to 0$, but I would like to know at what rate this function ...
1
vote
0answers
14 views

Convergence in distribution of the $2SLS$ estimator for some $\pi = \frac{h}{\sqrt{n}}$

Consider the following IV model. $$y_{1,i} = y_{2,i} \beta + u_i$$ $$y_{2,i} = z_{i}' \pi + v_{2,i}$$ with $(z_i', u_i, v_{2,i})$ iid and $\begin{pmatrix} u_i \\ v_{2,i} \end{pmatrix} \sim N(0_{...
2
votes
1answer
37 views

What is a good aproximation in asymptotic normality?

I have a conceptual doubt. For example, suppose I have $X_i \stackrel{iid}{\sim} N(\theta^*,1)$ and I know that (I have the information) $\theta^{*}\geq 0$. So I have the Constrained Maximum ...
1
vote
1answer
34 views

Implications of i.i.d. sample

I have the following question: I have managed to solve it but I wasn't sure if my reasoning was correct. So I can express the OLS estimator as $\sqrt{n}(\hat{\beta} - \beta) = (\frac{1}{n}\sum_{=1}^{...
0
votes
1answer
26 views

asymptotic behavior of OLS estimators

Conceptually when do estimated regression coefficients converge with the true values; is it a) when the sample size of observations tends to infinity or b) when the regression with the same predictors ...
2
votes
2answers
65 views

Can statistical dependence arise only at the limit?

(This has been inspired by a comment exchange with @guy). Assume we have two infinite sequences of random variables, $\{X_n\}$ and $\{Y_n\}$. Assume the RVs in $\{X_n\}$ are statistically ...
8
votes
3answers
146 views

When does $X_n\stackrel{d}{\rightarrow}X$ and $Y_n\stackrel{d}{\rightarrow}Y$ imply $X_n+Y_n\stackrel{d}{\rightarrow}X+Y$?

The question: $X_n\stackrel{d}{\rightarrow}X$ and $Y_n\stackrel{d}{\rightarrow}Y \stackrel{?}{\implies} X_n+Y_n\stackrel{d}{\rightarrow}X+Y$ I know that this does not hold in general; Slutsky's ...
4
votes
1answer
92 views

How is it possible for both the likelihood and log-likelihood to be asymptotically normal?

I was trying to understand asymptotic normality of the posterior better, and came across a confusing point. So let's say we have a likelihood, $L(\theta | X) = \Pi_{i=1}^n p(X_i | \theta)$, so the log-...
4
votes
1answer
82 views

distribution for scaled Maximum of n independent Weibulls for $n \to \infty$

Assume that $X_1, X_2,...\sim Weibull(\lambda, k) \quad iid.$, i.e. $F(X_1\leq x) = 1-e^{-(\lambda x)^k}$ define $M_n:= \max\{X_1, ..., X_n\}$ and $\tilde{M}_n:=\frac{M_n-b_n}{a_n}$ according to ...
2
votes
2answers
41 views

Asymptotic Expectation of Ratio of Sample Averages

I have two random variables: $X$ and $Y$. I know that: \begin{equation} E[X]=E[Y]=\mu>0 \end{equation} I know that variance of both can be bounded: \begin{equation} \operatorname{Var}[X]<k, \...
3
votes
0answers
68 views

Terminology: “Central Limit Theorem” for Delta Method

This is a question about when is it appropriate to call an asymptotic normality statement, the "Central Limit Theorem" (CLT). More specifically, suppose I have $X_1, X_2, \dots X_n$ iid from a ...
1
vote
0answers
27 views

why is method of moments estimates asymptotically normal

I have noticed that a lot of statistics textbooks contain lengthy discussions and detailed proofs on showing that MLE estimates are asymptotically normal (under regularity conditions). On the other ...
3
votes
0answers
36 views

Approximating standard error that contains a parameter, by replacing the parameter with its estimate

I am a bit confused about the following step I have seen in the stats literature which seems to me a bit circular. Say you are approximating the standard error of the MoM estimate of an exponential ...
3
votes
0answers
32 views

Limit behavior of “weighted” Poisson Binomial distribution

Given $X_n \sim \operatorname{Binomial}(n, p_n)$ it is known by the Poisson Limit Theorem that as $n \to \infty$ that $$X_n \to \operatorname{Poisson}(np_n).$$ This can be generalized to hold for ...
1
vote
2answers
35 views

Question about expectation in OLS?

Consider the linear model $$y_i = x_i^T\beta + \epsilon_i.$$ In ordinary least squares it is assumed that the errors satisfy $E[\epsilon_i]=0$. This implies that that $\dfrac{X^T\epsilon}{n} \to 0$ ...
0
votes
0answers
12 views

Sandwich standard errors versus typical standard error estimation

Question: do sandwich estimators of the standard errors equal the typical estimation of the standard error IF the data was generated with constant variance in the residuals and as the sample size ...
2
votes
0answers
39 views

Incorrect computation in Knight and Fu (2000)?

I'm currently reading Knight and Fu's 2000 paper on the asymptotics of "Bridge" estimators with a particular focus on LASSO as a special case. In the proof of theorem 2, they make the claim that under ...
0
votes
1answer
21 views

Feature selection by lasso in two samples compared to one joint sample

Let's say you have two sets of features $X_1$ and $X_2$ together with a response variable $Y$. I wonder whether the two following procedures are identical asymptotically (or in finite samples) in ...
4
votes
0answers
26 views

Is there an asymptotic distribution expansion that does not go negative when truncated?

I am looking into a problem where I have a distribution that converges to the normal distribution as its parameters become large. I am looking at the rate of this convergence, and seeking to describe ...
0
votes
0answers
52 views

Parametric Bootstrap Central limit theorem non i i d

I am having paired data with missing values in a single arm. I am willing to use parametric bootstrap with specific quadratic tests to test the hypothesis of equality of means. My model is as follows:...
3
votes
1answer
84 views

Is the distribution of the logarithm of the mean of Bernoulli random variables ($\log \overline X$) still asymptotically normal?

Let $\overline X$ be the mean of a Bernoulli random variable (r.v.) $$\overline X = \frac{1}{n}\sum_{i=1}^{n} X_i$$ where $X_i \in \{0, 1\}$. So based on Central Limit Theoreom, $$\overline X \sim \...