Questions tagged [asymptotics]
Asymptotic theory studies the properties of estimators and test statistics when the sample size approaches infinity.
0
votes
0answers
10 views
Hausman test statistic - to multiply or not to multiply by n
I am having some serious doubts regarding the formula of the Hausman statistic for the case in which I compare OLS and IV estimates.
I am getting confused with what my references are giving me.
What ...
0
votes
0answers
33 views
Limiting Distribution of $n\left[Y_n\right]$ where $Y_n$ is the minimum of a sample of size n from Uniform$\left(0,\theta\right)$ distribution
Suppose $X_1,X_2,\dots,X_n$ is a random sample from Uniform$(0,\theta)$ for some unknown $\theta > 0$. Let $Y_n$ be the minimum of $X_1,X_2,\dots,X_n$.
(a) Suppose $F_n$ is the CDF of $nY_n$. Show ...
0
votes
1answer
14 views
Why are oracle inequalities called that way?
The oracle property is an asymptotic property of an estimator, and is about variable selection:
An estimator $\hat \beta_n$ satisfies the oracle property if in the limit of $n\to \infty$, the ...
2
votes
0answers
31 views
Asymptotic normality of quadratic form?
Let $X$ be a $p$-dimensional vector that is asymptotically normal such that $$\sqrt{n}(X - \mu_X) \stackrel{d}\longrightarrow N(0, \Sigma)$$, and let $H$ be a random $p\times p$ symmetric matrix, ...
1
vote
0answers
39 views
Distribution of variance of $X(X^TX)^{-1}X^TZ$?
I had a question that I couldn't quite wrap my mind around.
Essentially, it's this. Suppose $x_i$ is a random $k$-vector, and $X$ is an $n\times k$ vector that is $n$ i.i.d. copies of this $x_i$ "...
1
vote
0answers
17 views
Counting the number of tuples $(i,i',k,k')$ satisfying some conditions
I'm struggling to find the cardinality of a set that will be presented below, as well as an upper bound for it. It is an engaging problem and perhaps not trivial.
I will give now the baseline of the ...
0
votes
0answers
24 views
Generalizing Bayesian methods by assuming a “distribution of distributions” instead of a prior
Bayesian methods assume a prior distribution with several hyperparameters. Unfortunately, this is asymptotically incorrect, because distributions in the real world are never exact. For example, the ...
0
votes
0answers
25 views
Finding a test using asymptotic theory. for Poisson $(\lambda)$
If we have a sample of Poisson $(\lambda)$
(a) Find a test for $H_0: \lambda =2$ vs $ H_a: \lambda =\lambda_1> 2$
(b) Find a test using asymptotic theory.
(c) Compare the results in en (a) y (...
2
votes
0answers
62 views
Asymptotic relationship between tests of compact composite nulls and one sided tests
A motivating example
Let $x_t\sim N(\mu,1)$ for $t=1,\dots,T$. Consider testing the null $H_0: \mu\in[-1,1]$ against the alternative $H_1:\mu\in\mathbb{R}\setminus [-1,1]$.
The natural (UMPU?) test ...
2
votes
0answers
27 views
Understanding simple LRT test asymptotic using Taylor expansion?
I am trying to understand the proof that the LRT test for
$$H_0: \theta = \theta_0 \quad vs \quad H_1: \theta \neq \theta_0$$
is asymptotically chisquared distributed with one degree of freedom. I ...
0
votes
0answers
14 views
confidence interval for the variance with unknown distribution
Let $X_i \sim \text{iid}(\mu, \sigma^2)$ (the question does not specify whether or not $\mu$ and $\sigma^2$ are known). I have to show the confidence interval for the variance. Since $X_i$ is not ...
0
votes
0answers
12 views
When is the bias of a statistic of the form a/n + b/n^2 + c/n^3 +
In many books the bias-correction of the Jackknife resampling method is being prooved under the assumption, that the bias has a special form, namely a/n + b/n^2 + c/n^3 * ...
Sometimes it's written "...
6
votes
2answers
109 views
What's an example of an estimator that is $o_p(n^{-1/2})$?
For $X_i$ i.i.d. normal with mean $\mu < \infty$ and variance 1, by the law of large numbers, we have $\bar{X} \overset{\mathcal{P}}{\rightarrow}\mu$, i.e. $\bar{X} - \mu = o_p(1)$. Similarly, by ...
1
vote
0answers
30 views
Approximating the Kullback-Leibler Divergence with a Laplace approximation
Suppose I wish to compute the (asymptotic) Kullback-Leibler Divergence (KLD) between the exact Bayesian posterior
$$q_{n}(\theta|x_{1:n}) \propto \pi(\theta)\prod_{i=1}^n p(x_i|\theta)$$
and the ...
0
votes
0answers
23 views
Bound of the bias of a Gaussian Process by its standard deviation in Gaussian Process Regression
In Gaussian Process Regression (GPR), intuitively, the bias of the conditioned Gaussian Process (posterior) at a location $x^*$ gets smaller if the variance at $x^*$ is getting smaller, for example in ...
9
votes
1answer
173 views
Is MLE of $\theta$ asymptotically normal when $(X,Y)\sim e^{-(x/\theta+\theta y)}\mathbf1_{x,y>0}$?
Suppose $(X,Y)$ has the pdf
$$f_{\theta}(x,y)=e^{-(x/\theta+\theta y)}\mathbf1_{x>0,y>0}\quad,\,\theta>0$$
Density of the sample $(\mathbf X,\mathbf Y)=(X_i,Y_i)_{1\le i\le n}$ drawn from ...
0
votes
1answer
44 views
Approximate distribution for sum of squares of standardized Poisson random variables
Suppose that $X_1, ..., X_n$ are independent and identically distributed Poisson($\lambda$) random variables.
What is a good approximating distribution for $\sum_{i = 1}^{200} \frac{(X_i - \lambda)^2}...
1
vote
1answer
33 views
Convergence of error term
I was deriving the ratio of a Laplace approximation with the true quantity and I got this:
$$
\left(n\bar{x} + \alpha - \frac{1}{2}\right)\log\left(\frac{\bar{x}+(a-1)/n}{\bar{x}+\alpha/n}\right) + n\...
0
votes
0answers
15 views
The Cochran and Cox Approximation Pair T test Unequal Variance
Hi everyone I hope you are well. Maybe as you know according to Behners-Fisher problem (unequal variance case of samples) there are some kind of approximations. I have recently covered the ...
0
votes
0answers
25 views
Convergence in probability (asymptotic notation) result
Let $h=h_n$ be a sequence of numbers such that $h_n \rightarrow 0$ as $n \rightarrow \infty$, $\mu$ be a real constant and $f$ be some probability density function. I was wondering if the following ...
0
votes
0answers
18 views
The limit distribution of Wilcoxon signed rank statistic?
An alternative representation of the Wilcoxon signed rank statistic $V$ is
$V=\sum_{i\le j}\mathbb{I}_{\{X_i+X_j>0\}}=\sum_i\mathbb{I}_{\{X_i>0\}}+\sum_{i<j}\mathbb{I}_{\{X_i+X_j>0\}}$
...
3
votes
1answer
68 views
AIC for increasing sample size
I am using AIC as a model selection criteria in one of my projects. However, since AIC isn't dependent on the number of points sampled, for large n the log likelihood term rapidly outscales the ...
3
votes
2answers
132 views
What “nice” property of a confidence set is being violated
Suppose that I have $X_{i} \overset{i.i.d.}{\sim} P$ with $E[X_{i}]=\mu$ and $V[X_{i}^{2}] = \sigma^{2}<\infty$.
Then by the central limit theorem I know that:
\begin{align}
\sqrt{n} (\bar{X}_{n} ...
0
votes
0answers
19 views
Behavior of likelihood far from peak
In the neighborhood of the maximum likelihood point the log-likelihood function is often fruitfully expanded as a quadratic function of the parameters.
Are there any general results about the shape ...
2
votes
1answer
128 views
How to derive the asymptotic distribution of t-statistic?
Let ${X_n}$ be an IID sample such that ${X_i} \sim N(\mu,\sigma^2)$. When both $\mu$ and $\sigma$ are unknown, we construct $t(\hat{\mu},s)=\dfrac{\sqrt{n}(\hat{\mu}-\mu)}{s}$, where $s$ is the sample ...
4
votes
1answer
70 views
Convergence of scaled $L_1$ distance between two sorted random vectors with same limiting distribution
Let $ X=(X_1,\dots,X_n) $ and $ Y=(Y_1,\dots,Y_n) $ where the RVs $ X_1,\dots X_n, Y_1,\dots Y_n $ are independent and have the same limiting distribution (assume for simplicity that all moments exist)...
1
vote
0answers
20 views
Type 2 error in t-test on time series
I have an AR(1) time series with $1>\phi>0$. If I naively use t-test to check $H_0:\mu=0$ and it does not reject the null, then can I accept the result?
I think yes because for a time-series ...
0
votes
0answers
18 views
Asymptotic probability that chi-squared distribution exceeds its mean
Let $\chi_k$ be the chi-squared distribution with $k$ degrees of freedom. $\chi_k$ has mean $k$. Is it true that
$$\lim_{k \to \infty} \Pr[\chi_k \geq k] = 1/2$$
This seems pretty plausible.
0
votes
0answers
27 views
Consistency of variance estimator in OLS [duplicate]
Given the model,
$$
y_i = x_i'\beta + \epsilon_i \quad \epsilon_i \sim N(0, \sigma^2)
\quad iid \quad \forall i = 1, ..,n
$$
how can I prove that the estimator of the variance $\hat{\sigma}^2 = \...
0
votes
0answers
33 views
Tail behaviour of normal cdf?
Q: What is the tail behaviour of $\log \Phi(t)$ as $t \to \infty$?
Since $\Phi(t) \to 1$ as $t \to \infty$, we know that $\log \Phi(t)\to 0$, but I would like to know at what rate this function ...
1
vote
0answers
14 views
Convergence in distribution of the $2SLS$ estimator for some $\pi = \frac{h}{\sqrt{n}}$
Consider the following IV model.
$$y_{1,i} = y_{2,i} \beta + u_i$$
$$y_{2,i} = z_{i}' \pi + v_{2,i}$$
with $(z_i', u_i, v_{2,i})$ iid and $\begin{pmatrix}
u_i \\
v_{2,i}
\end{pmatrix} \sim N(0_{...
2
votes
1answer
37 views
What is a good aproximation in asymptotic normality?
I have a conceptual doubt. For example, suppose I have $X_i \stackrel{iid}{\sim} N(\theta^*,1)$ and I know that (I have the information) $\theta^{*}\geq 0$. So I have the Constrained Maximum ...
1
vote
1answer
34 views
Implications of i.i.d. sample
I have the following question:
I have managed to solve it but I wasn't sure if my reasoning was correct. So I can express the OLS estimator as
$\sqrt{n}(\hat{\beta} - \beta) = (\frac{1}{n}\sum_{=1}^{...
0
votes
1answer
26 views
asymptotic behavior of OLS estimators
Conceptually when do estimated regression coefficients converge with the true values; is it
a) when the sample size of observations tends to infinity
or
b) when the regression with the same predictors ...
2
votes
2answers
65 views
Can statistical dependence arise only at the limit?
(This has been inspired by a comment exchange with @guy).
Assume we have two infinite sequences of random variables, $\{X_n\}$ and $\{Y_n\}$. Assume the RVs in $\{X_n\}$ are statistically ...
8
votes
3answers
146 views
When does $X_n\stackrel{d}{\rightarrow}X$ and $Y_n\stackrel{d}{\rightarrow}Y$ imply $X_n+Y_n\stackrel{d}{\rightarrow}X+Y$?
The question:
$X_n\stackrel{d}{\rightarrow}X$ and $Y_n\stackrel{d}{\rightarrow}Y \stackrel{?}{\implies} X_n+Y_n\stackrel{d}{\rightarrow}X+Y$
I know that this does not hold in general; Slutsky's ...
4
votes
1answer
92 views
How is it possible for both the likelihood and log-likelihood to be asymptotically normal?
I was trying to understand asymptotic normality of the posterior better, and came across a confusing point. So let's say we have a likelihood, $L(\theta | X) = \Pi_{i=1}^n p(X_i | \theta)$, so the log-...
4
votes
1answer
82 views
distribution for scaled Maximum of n independent Weibulls for $n \to \infty$
Assume that $X_1, X_2,...\sim Weibull(\lambda, k) \quad iid.$, i.e.
$F(X_1\leq x) = 1-e^{-(\lambda x)^k}$
define $M_n:= \max\{X_1, ..., X_n\}$ and $\tilde{M}_n:=\frac{M_n-b_n}{a_n}$
according to ...
2
votes
2answers
41 views
Asymptotic Expectation of Ratio of Sample Averages
I have two random variables: $X$ and $Y$. I know that:
\begin{equation}
E[X]=E[Y]=\mu>0
\end{equation}
I know that variance of both can be bounded:
\begin{equation}
\operatorname{Var}[X]<k, \...
3
votes
0answers
68 views
Terminology: “Central Limit Theorem” for Delta Method
This is a question about when is it appropriate to call an asymptotic normality statement, the "Central Limit Theorem" (CLT).
More specifically, suppose I have $X_1, X_2, \dots X_n$ iid from a ...
1
vote
0answers
27 views
why is method of moments estimates asymptotically normal
I have noticed that a lot of statistics textbooks contain lengthy discussions and detailed proofs on showing that MLE estimates are asymptotically normal (under regularity conditions).
On the other ...
3
votes
0answers
36 views
Approximating standard error that contains a parameter, by replacing the parameter with its estimate
I am a bit confused about the following step I have seen in the stats literature which seems to me a bit circular.
Say you are approximating the standard error of the MoM estimate of an exponential ...
3
votes
0answers
32 views
Limit behavior of “weighted” Poisson Binomial distribution
Given $X_n \sim \operatorname{Binomial}(n, p_n)$ it is known by the Poisson Limit Theorem that as $n \to \infty$ that $$X_n \to \operatorname{Poisson}(np_n).$$
This can be generalized to hold for ...
1
vote
2answers
35 views
Question about expectation in OLS?
Consider the linear model
$$y_i = x_i^T\beta + \epsilon_i.$$
In ordinary least squares it is assumed that the errors satisfy $E[\epsilon_i]=0$. This implies that that $\dfrac{X^T\epsilon}{n} \to 0$ ...
0
votes
0answers
12 views
Sandwich standard errors versus typical standard error estimation
Question: do sandwich estimators of the standard errors equal the typical estimation of the standard error IF the data was generated with constant variance in the residuals and as the sample size ...
2
votes
0answers
39 views
Incorrect computation in Knight and Fu (2000)?
I'm currently reading Knight and Fu's 2000 paper on the asymptotics of "Bridge" estimators with a particular focus on LASSO as a special case. In the proof of theorem 2, they make the claim that under ...
0
votes
1answer
21 views
Feature selection by lasso in two samples compared to one joint sample
Let's say you have two sets of features $X_1$ and $X_2$ together with a response variable $Y$. I wonder whether the two following procedures are identical asymptotically (or in finite samples) in ...
4
votes
0answers
26 views
Is there an asymptotic distribution expansion that does not go negative when truncated?
I am looking into a problem where I have a distribution that converges to the normal distribution as its parameters become large. I am looking at the rate of this convergence, and seeking to describe ...
0
votes
0answers
52 views
Parametric Bootstrap Central limit theorem non i i d
I am having paired data with missing values in a single arm. I am willing to use parametric bootstrap with specific quadratic tests to test the hypothesis of equality of means. My model is as follows:...
3
votes
1answer
84 views
Is the distribution of the logarithm of the mean of Bernoulli random variables ($\log \overline X$) still asymptotically normal?
Let $\overline X$ be the mean of a Bernoulli random variable (r.v.)
$$\overline X = \frac{1}{n}\sum_{i=1}^{n} X_i$$
where $X_i \in \{0, 1\}$. So based on Central Limit Theoreom,
$$\overline X \sim \...
