Questions tagged [asymptotics]
Asymptotic theory studies the properties of estimators and test statistics when the sample size approaches infinity.
487
questions
0
votes
0answers
14 views
Limiting distribution of sample variance and standard deviation
I have a centered Gaussian sample of $n$ elements $X_i,\,i=1,..,n$, with variance $\sigma^2$. I would like to find the limiting distribution of the sample variance $\sigma_n^2=\frac 1n \sum_{i=1}^n ...
0
votes
0answers
25 views
Why do some people say that an asymptotically unbiased estimator “satisfies a strong law of large numbers”?
If $x\in\mathbb R$, an estimator for $x$ is an integrable random variable $X$. We say that $X$ is unbiased if $\operatorname{Bias}(x,X):=x-\operatorname E[X]=0$.
Now, in the context of Markov chain ...
0
votes
0answers
9 views
What is gained by comparing the higher-order asymptotic bias of estimators?
A paper I'm reading says:
We characterize the $N^{-1}$ order asymptotic bias of [the estimator
being proposed in the paper] and [some other estimators] under
conditions where they are first-...
0
votes
0answers
3 views
Goodness of fit between edf and cdf w.r.t to Cramer-von-Mises distance
For
$$
h(F) = E_G \left [ (F(X) - G(X))^2 \right]
$$
I try to derive that the empirical distribution $F_n$ has the goodness of fit
$$
h(F_n) = \frac{1}{n} \sum_{i=1}^n \left [ G(X_{(i)}) - \frac{i-1/...
0
votes
0answers
26 views
Explain asymptotic optimality property of AIC (Akaike information criterion)
I am trying to use AIC in my research, I know how to apply it and how to interpret its value, but I do not understand the asymptotic optimality property in general and why AIC has this property. Can ...
1
vote
1answer
32 views
Asymptotic normality: proof strategy
Given a estimator $\hat \theta$ of $\theta$, I want to show that $\sqrt{n}(\hat\theta -\theta-B)\to N(0,V_\theta)$ as $n\to\infty$, given that the limit $V_\theta$ exists and $B>0$ possibly ...
3
votes
0answers
44 views
Reduce the asymptotic variance for a class of Metropolis-Hasting estimates
I'm running the Metropolis-Hastings algorithm with state space $E$, target distribution $\mu=p\lambda$ and proposal kernel $Q$ to estimate $\mu(hf)$ for a fixed function $f:E\to[0,\infty)^3$ and a ...
2
votes
1answer
53 views
Kalman filter: asymptotic of state estimate
Assume we have a linear state-space model:
$$
z_{k} = Hx_{k} + v_{k}\\
x_{k} = F x_{k-1} + Bu + w_{k},
$$
where $u$ is some control variable (constant intercept is the simplest case).
Kalman filter, ...
1
vote
0answers
31 views
Asymptotic Distribution of Wald test
If $X_1 , X_2 , ... X_n$ are iid and they have the same pdf $f_θ(x)$ .
Consider testing $H_0 : θ= θ_0$ vs $H_n : θ = θ_0 +Δ * n^{-1/2}$.
where , $Δ = (Δ_1 , Δ_2 , .. Δ_k)^T$
We want to find the ...
3
votes
0answers
59 views
Generalized univariate normal distribution with $k+1$ parameters
Final update on 11/28/2019: I have worked on this a bit more, and wrote an article summarizing all the main findings. You can read it here.
The goal here is to obtain a highly generic family of ...
1
vote
1answer
18 views
Show that bias term involving an indicator function convergences to zero
Assume that we have $N$ observations of i.i.d. data $(Y_i,X_i)_{i=1}^{N}$. We want to learn the model given by $Y=f(X)+\epsilon$. We use the data to estimate $\hat{f}$ using any machine learning ...
1
vote
0answers
10 views
Rate of convergence of variance of kernel averages
I'm reading Hansen's (2008, p. 729) Theorem 1 where he bounds the variance of averages of the form
$$\hat\Psi(x)=\frac{1}{Th}\sum_{t=1}^T Y_t K\bigg(\frac{x-X_t}{h}\bigg)$$
given that $\{(Y_t,X_t)\}_{...
3
votes
0answers
37 views
Asymptotic behaviour of order statistic $x_{(n-k+1)}$ when k is $n^{\alpha}$
I am interested in the asymptotic behavior of the top k-th order statistic
$x_{(n-k+1)}$ from n i.i.d. standard normal samples, when k is $n^\alpha$ where $\alpha\in (0,1)$.
I just wonder if we can ...
1
vote
0answers
76 views
Estimator, Bias and asymptotic distribution
I have a model;
$$y_i = \beta_1 + \frac{1}{\beta_2}x_i+\epsilon_i$$
To simplify I use OLS to regress on;
$$y_i = \delta_1 + \delta_2 x_1 + \epsilon_i$$
Thus I obtain the two estimators $\hat{\...
1
vote
2answers
96 views
Asymptotic distribution of $\frac{\overline x_n+ \overline y_n}{\overline x_n- \overline y_n}$
Let $x_1, \ldots, x_n$ and $y_1, \ldots, y_n$ be two independent random samples from $X$ and $Y$. We have $\mu_X = E (X ) > 0, \mu_Y = E (Y ) > 0$ and $\sigma^2_X = Var (X )$ and $\sigma^2_Y = ...
3
votes
2answers
73 views
Consistency of MLE of $\alpha$ when pdf is $f(x;\alpha,\beta)=\frac{\alpha x^{\alpha-1}}{\beta^\alpha}1_{0<x<\beta}$
I have a sample of size $n$ from the following distribution:
$$f(x;\alpha,\beta)=\frac{\alpha x^{\alpha-1}}{\beta^\alpha}1_{0<x<\beta}\quad,\,\alpha>0$$
I found that the MLEs are
$$\hat{\...
0
votes
0answers
11 views
Asymptotic Distribution of ratio of two autovariance with MA(1) model
I want to know the asymptotic distribution of
r(2)/r(1) where r(2) is autocovariance of MA(1) with lag 2
and r(1) is autocovariance of MA(1) with lag 1
Could you give me some hints to solve this?
(...
0
votes
0answers
63 views
Asymptotic behaviour of logistic regression [duplicate]
Assume that we use logistic regression with a given regression function r(x) for each sample vector x. Is is true that if we have infinite samples, we have r(x) = log(p(x)) - log(q(x)) where p(x) is ...
2
votes
1answer
54 views
Which transformation needed to make variance independent of population parameter?
Suppose $s^2$ is the sample variance of a sample$(\text{of size }n)$ from a normal population with mean $\mu$ and variance $\sigma^2. \text{Here }s^2=\frac{\sum_{i=1}^n(x_i-\overline{x})^2}{n-1} $
As ...
0
votes
0answers
16 views
Sampling distribution of the Score statistics of a GLM MODEL
In the context of a GLM (with a distribution that belongs to the exponential family), we often compute the score statistics
$$
U = \frac{\partial LogLike(\boldsymbol{\beta};\mathbf{y})}{\partial\...
1
vote
0answers
17 views
Markov inequality and Boundness in probability
Let $\{X_n\}$ and $\{a_n\}$ be sequences of random variables and real numbers, respectively.
Say that $X_n=O_P(a_n)$ iff $\forall\epsilon>0$, $\exists N,M>0$ such that for all $n>N$, we ...
2
votes
0answers
30 views
Inadmissible MLE
Let us say, I have a distribution with some unknown parameter and I find the MLE of the parameter. I prove that the MLE is inadmissible using some other estimator. Why is this not in a contradiction ...
0
votes
1answer
35 views
Asymptotic distribution of an estimator [duplicate]
I have the following problem:
Let $X_1,..., X_n$ be a sample of independent, identically distributed random variables, with density $$f_{\theta}(x)=\begin{cases}
e^{\theta-x}, & \text{if } x\geq \...
1
vote
0answers
7 views
Asymptotic distribution of intraclass correlation
Consider the random effects ANOVA model below (notation based on Snijders and Bosker, 1999), where $j$ represents a group and $i$ an individual:
$$
Y_{ij} = \mu + U_j + R_{ij}, \qquad var(Y_{ij}) = \...
1
vote
0answers
39 views
Difference between predictions of the OLS model and a leave-one-out model
Consider OLS regression with the true model $y = {\theta^{*}}^{\text{T}} x + \varepsilon$, where $x$ denotes the (deterministic) independent variables, $y$ denotes the dependent (random) variable, and ...
2
votes
0answers
24 views
Minimization of the asymptotic variance in MCMC
Suppose $(X_n)_{n\in\mathbb N_0}$ is a Markov chain generated by the Metropolis-Hastings algorithm. Assume $(X_n)_{n\in\mathbb N_0}$ is stationary and consider the ergodic averages $$A_n:=\frac1n\sum_{...
1
vote
3answers
97 views
Parameter estimation from single dynamic experiment. How many degrees of freedom?
I am estimating a parameter (diffusivity) from a set of experimental data coming from the same dynamic experiment: it can be summarised as changing the oxygen partial pressure around a solid and ...
1
vote
0answers
46 views
Convergence in distribution of $\sqrt{n-3}(z -\zeta)$
Let $(x_i,y_i)$ be iid $(x,y), i = 1,..,n$ be any bivariate sample. The correlation coefficient
\begin{equation}
\begin{split}
\rho & = \text{cor}(x,y) \\
& = \frac{\text{cov}(x,y)}{\sqrt{\...
2
votes
1answer
35 views
Asymptotic Distribution of Covariance
I've seen a lot of questions revolving around the asymptotic distribution regarding the sample variance, such as $\sqrt{n} (s^2 - \sigma^2) \xrightarrow{d} N(0, \mu_4 - \sigma^4)$, however, what would ...
0
votes
1answer
54 views
Asymptotic Distribution of Minimum Uniform Random Variables
I've been working on this problem for a while, and I've made some progress, but I'm still stuck on some parts. I was hoping to get some assistance with this!
Let $M_n = \min(X_1, ..., X_n)$ where $...
2
votes
1answer
42 views
What is the variance of the estimator in ordinary least squares with correlated residuals
If we assumed that $y \sim N(X\beta,S)$ where
S=
$\sigma^2\begin{bmatrix}
1 & \rho & \rho &...\\
\rho & 1 & \rho &...\\
\rho & \rho & 1 &...\\
\rho & \rho &...
0
votes
0answers
27 views
Comparing dimensionality
I am reading a couple of papers,
The first one claims that $p_n = O(n^a)$ for some $0 \leq a < \infty$
The second paper claims $p_n = o(\exp(n^\varepsilon))$ for some $0< \varepsilon < .2$ ...
0
votes
0answers
86 views
Variance reduction of an estimator arising from the marginal destribution of a Metropolis-Hastings chain
Let
$(E,\mathcal E,\lambda)$ and $(E',\mathcal E',\lambda')$ be measure spaces
$f\in L^2(\lambda)$
$I$ be a finite nonempty set
$\varphi_i:E'\to E$ be bijective $(\mathcal E',\mathcal E)$-measurable ...
0
votes
0answers
51 views
Maximal inequality for quadratic forms as functions of matrices?
In general, an $M$-estimator is defined as a maximizer of some objective function $Q_n(\theta)$: $\hat{\theta} = \arg\max_{\theta} Q_n(\theta)$. Suppose that $q(\theta) = E Q_n(\theta)$ is uniquely ...
0
votes
0answers
21 views
Fitting a regression model with spurious variables: do they vanish with large samples?
I am fitting the usual linear regression model
$$y_j = x_j^T\beta + e_j,$$
where the errors $e_j$ are iid normal with unknown variance. If the vector of covariates $x_j$ contain spurious variables, ...
0
votes
1answer
29 views
How are Wald type tests better asymptotically? [closed]
I heard that Wald tests are better asymptotically than other tests. But what does this mean? Does it mean that it's better at testing for population level differences? Is it better for "big data" and ...
2
votes
0answers
24 views
Asymptotic approximation of log-probability using first four moments
Consider a random variable $X \sim p_{n,\theta}$ where the first four moments are given by known functions:
$$\begin{matrix}
\ \ \ \ \ \ \mathbb{E}(X) \equiv \mu(n,\theta) & & & \ \ \ \ \ ...
0
votes
0answers
34 views
Martingale Difference Sequence CLT
Could you provide me with the proof of the following:
$$n^{-1/2} \cdot\sum_t a_{t-j}e_t$$ converges to normal distribution as $n$ goes infinity by martingale difference sequence CLT where $a_t = \...
3
votes
2answers
97 views
Asymptotic normality of random vector
If each component of a random vector is asymptotically normal by the central limit theorem (CLT), can anything be said about the asymptotic distribution of the random vector? The components are not ...
0
votes
0answers
21 views
Hausman test statistic - to multiply or not to multiply by n
I am having some serious doubts regarding the formula of the Hausman statistic for the case in which I compare OLS and IV estimates.
I am getting confused with what my references are giving me.
What ...
0
votes
0answers
45 views
Limiting Distribution of $n\left[Y_n\right]$ where $Y_n$ is the minimum of a sample of size n from Uniform$\left(0,\theta\right)$ distribution
Suppose $X_1,X_2,\dots,X_n$ is a random sample from Uniform$(0,\theta)$ for some unknown $\theta > 0$. Let $Y_n$ be the minimum of $X_1,X_2,\dots,X_n$.
(a) Suppose $F_n$ is the CDF of $nY_n$. Show ...
0
votes
1answer
25 views
Why are oracle inequalities called that way?
The oracle property is an asymptotic property of an estimator, and is about variable selection:
An estimator $\hat \beta_n$ satisfies the oracle property if in the limit of $n\to \infty$, the ...
2
votes
0answers
59 views
Asymptotic normality of quadratic form?
Let $X$ be a $p$-dimensional vector that is asymptotically normal such that $$\sqrt{n}(X - \mu_X) \stackrel{d}\longrightarrow N(0, \Sigma)$$, and let $H$ be a random $p\times p$ symmetric matrix, ...
1
vote
0answers
42 views
Distribution of variance of $X(X^TX)^{-1}X^TZ$?
I had a question that I couldn't quite wrap my mind around.
Essentially, it's this. Suppose $x_i$ is a random $k$-vector, and $X$ is an $n\times k$ vector that is $n$ i.i.d. copies of this $x_i$ "...
1
vote
0answers
19 views
Counting the number of tuples $(i,i',k,k')$ satisfying some conditions
I'm struggling to find the cardinality of a set that will be presented below, as well as an upper bound for it. It is an engaging problem and perhaps not trivial.
I will give now the baseline of the ...
1
vote
1answer
39 views
Generalizing Bayesian methods by assuming a “distribution of distributions” instead of a prior
Bayesian methods assume a prior distribution with several hyperparameters. Unfortunately, this is asymptotically incorrect, because distributions in the real world are never exact. For example, the ...
0
votes
0answers
28 views
Finding a test using asymptotic theory. for Poisson $(\lambda)$
If we have a sample of Poisson $(\lambda)$
(a) Find a test for $H_0: \lambda =2$ vs $ H_a: \lambda =\lambda_1> 2$
(b) Find a test using asymptotic theory.
(c) Compare the results in en (a) y (...
2
votes
0answers
94 views
Asymptotic relationship between tests of compact composite nulls and one sided tests
A motivating example
Let $x_t\sim N(\mu,1)$ for $t=1,\dots,T$. Consider testing the null $H_0: \mu\in[-1,1]$ against the alternative $H_1:\mu\in\mathbb{R}\setminus [-1,1]$.
The natural (UMPU?) test ...
3
votes
0answers
46 views
Understanding simple LRT test asymptotic using Taylor expansion?
I am trying to understand the proof that the LRT test for
$$H_0: \theta = \theta_0 \quad vs \quad H_1: \theta \neq \theta_0$$
is asymptotically chisquared distributed with one degree of freedom. I ...
0
votes
0answers
24 views
confidence interval for the variance with unknown distribution
Let $X_i \sim \text{iid}(\mu, \sigma^2)$ (the question does not specify whether or not $\mu$ and $\sigma^2$ are known). I have to show the confidence interval for the variance. Since $X_i$ is not ...
