Questions tagged [asymptotics]

Asymptotic theory studies the properties of estimators and test statistics when the sample size approaches infinity.

Filter by
Sorted by
Tagged with
0
votes
0answers
14 views

Limiting distribution of sample variance and standard deviation

I have a centered Gaussian sample of $n$ elements $X_i,\,i=1,..,n$, with variance $\sigma^2$. I would like to find the limiting distribution of the sample variance $\sigma_n^2=\frac 1n \sum_{i=1}^n ...
0
votes
0answers
25 views

Why do some people say that an asymptotically unbiased estimator “satisfies a strong law of large numbers”?

If $x\in\mathbb R$, an estimator for $x$ is an integrable random variable $X$. We say that $X$ is unbiased if $\operatorname{Bias}(x,X):=x-\operatorname E[X]=0$. Now, in the context of Markov chain ...
0
votes
0answers
9 views

What is gained by comparing the higher-order asymptotic bias of estimators?

A paper I'm reading says: We characterize the $N^{-1}$ order asymptotic bias of [the estimator being proposed in the paper] and [some other estimators] under conditions where they are first-...
0
votes
0answers
3 views

Goodness of fit between edf and cdf w.r.t to Cramer-von-Mises distance

For $$ h(F) = E_G \left [ (F(X) - G(X))^2 \right] $$ I try to derive that the empirical distribution $F_n$ has the goodness of fit $$ h(F_n) = \frac{1}{n} \sum_{i=1}^n \left [ G(X_{(i)}) - \frac{i-1/...
0
votes
0answers
26 views

Explain asymptotic optimality property of AIC (Akaike information criterion)

I am trying to use AIC in my research, I know how to apply it and how to interpret its value, but I do not understand the asymptotic optimality property in general and why AIC has this property. Can ...
1
vote
1answer
32 views

Asymptotic normality: proof strategy

Given a estimator $\hat \theta$ of $\theta$, I want to show that $\sqrt{n}(\hat\theta -\theta-B)\to N(0,V_\theta)$ as $n\to\infty$, given that the limit $V_\theta$ exists and $B>0$ possibly ...
3
votes
0answers
44 views

Reduce the asymptotic variance for a class of Metropolis-Hasting estimates

I'm running the Metropolis-Hastings algorithm with state space $E$, target distribution $\mu=p\lambda$ and proposal kernel $Q$ to estimate $\mu(hf)$ for a fixed function $f:E\to[0,\infty)^3$ and a ...
2
votes
1answer
53 views

Kalman filter: asymptotic of state estimate

Assume we have a linear state-space model: $$ z_{k} = Hx_{k} + v_{k}\\ x_{k} = F x_{k-1} + Bu + w_{k}, $$ where $u$ is some control variable (constant intercept is the simplest case). Kalman filter, ...
1
vote
0answers
31 views

Asymptotic Distribution of Wald test

If $X_1 , X_2 , ... X_n$ are iid and they have the same pdf $f_θ(x)$ . Consider testing $H_0 : θ= θ_0$ vs $H_n : θ = θ_0 +Δ * n^{-1/2}$. where , $Δ = (Δ_1 , Δ_2 , .. Δ_k)^T$ We want to find the ...
3
votes
0answers
59 views

Generalized univariate normal distribution with $k+1$ parameters

Final update on 11/28/2019: I have worked on this a bit more, and wrote an article summarizing all the main findings. You can read it here. The goal here is to obtain a highly generic family of ...
1
vote
1answer
18 views

Show that bias term involving an indicator function convergences to zero

Assume that we have $N$ observations of i.i.d. data $(Y_i,X_i)_{i=1}^{N}$. We want to learn the model given by $Y=f(X)+\epsilon$. We use the data to estimate $\hat{f}$ using any machine learning ...
1
vote
0answers
10 views

Rate of convergence of variance of kernel averages

I'm reading Hansen's (2008, p. 729) Theorem 1 where he bounds the variance of averages of the form $$\hat\Psi(x)=\frac{1}{Th}\sum_{t=1}^T Y_t K\bigg(\frac{x-X_t}{h}\bigg)$$ given that $\{(Y_t,X_t)\}_{...
3
votes
0answers
37 views

Asymptotic behaviour of order statistic $x_{(n-k+1)}$ when k is $n^{\alpha}$

I am interested in the asymptotic behavior of the top k-th order statistic $x_{(n-k+1)}$ from n i.i.d. standard normal samples, when k is $n^\alpha$ where $\alpha\in (0,1)$. I just wonder if we can ...
1
vote
0answers
76 views

Estimator, Bias and asymptotic distribution

I have a model; $$y_i = \beta_1 + \frac{1}{\beta_2}x_i+\epsilon_i$$ To simplify I use OLS to regress on; $$y_i = \delta_1 + \delta_2 x_1 + \epsilon_i$$ Thus I obtain the two estimators $\hat{\...
1
vote
2answers
96 views

Asymptotic distribution of $\frac{\overline x_n+ \overline y_n}{\overline x_n- \overline y_n}$

Let $x_1, \ldots, x_n$ and $y_1, \ldots, y_n$ be two independent random samples from $X$ and $Y$. We have $\mu_X = E (X ) > 0, \mu_Y = E (Y ) > 0$ and $\sigma^2_X = Var (X )$ and $\sigma^2_Y = ...
3
votes
2answers
73 views

Consistency of MLE of $\alpha$ when pdf is $f(x;\alpha,\beta)=\frac{\alpha x^{\alpha-1}}{\beta^\alpha}1_{0<x<\beta}$

I have a sample of size $n$ from the following distribution: $$f(x;\alpha,\beta)=\frac{\alpha x^{\alpha-1}}{\beta^\alpha}1_{0<x<\beta}\quad,\,\alpha>0$$ I found that the MLEs are $$\hat{\...
0
votes
0answers
11 views

Asymptotic Distribution of ratio of two autovariance with MA(1) model

I want to know the asymptotic distribution of r(2)/r(1) where r(2) is autocovariance of MA(1) with lag 2 and r(1) is autocovariance of MA(1) with lag 1 Could you give me some hints to solve this? (...
0
votes
0answers
63 views

Asymptotic behaviour of logistic regression [duplicate]

Assume that we use logistic regression with a given regression function r(x) for each sample vector x. Is is true that if we have infinite samples, we have r(x) = log(p(x)) - log(q(x)) where p(x) is ...
2
votes
1answer
54 views

Which transformation needed to make variance independent of population parameter?

Suppose $s^2$ is the sample variance of a sample$(\text{of size }n)$ from a normal population with mean $\mu$ and variance $\sigma^2. \text{Here }s^2=\frac{\sum_{i=1}^n(x_i-\overline{x})^2}{n-1} $ As ...
0
votes
0answers
16 views

Sampling distribution of the Score statistics of a GLM MODEL

In the context of a GLM (with a distribution that belongs to the exponential family), we often compute the score statistics $$ U = \frac{\partial LogLike(\boldsymbol{\beta};\mathbf{y})}{\partial\...
1
vote
0answers
17 views

Markov inequality and Boundness in probability

Let $\{X_n\}$ and $\{a_n\}$ be sequences of random variables and real numbers, respectively. Say that $X_n=O_P(a_n)$ iff $\forall\epsilon>0$, $\exists N,M>0$ such that for all $n>N$, we ...
2
votes
0answers
30 views

Inadmissible MLE

Let us say, I have a distribution with some unknown parameter and I find the MLE of the parameter. I prove that the MLE is inadmissible using some other estimator. Why is this not in a contradiction ...
0
votes
1answer
35 views

Asymptotic distribution of an estimator [duplicate]

I have the following problem: Let $X_1,..., X_n$ be a sample of independent, identically distributed random variables, with density $$f_{\theta}(x)=\begin{cases} e^{\theta-x}, & \text{if } x\geq \...
1
vote
0answers
7 views

Asymptotic distribution of intraclass correlation

Consider the random effects ANOVA model below (notation based on Snijders and Bosker, 1999), where $j$ represents a group and $i$ an individual: $$ Y_{ij} = \mu + U_j + R_{ij}, \qquad var(Y_{ij}) = \...
1
vote
0answers
39 views

Difference between predictions of the OLS model and a leave-one-out model

Consider OLS regression with the true model $y = {\theta^{*}}^{\text{T}} x + \varepsilon$, where $x$ denotes the (deterministic) independent variables, $y$ denotes the dependent (random) variable, and ...
2
votes
0answers
24 views

Minimization of the asymptotic variance in MCMC

Suppose $(X_n)_{n\in\mathbb N_0}$ is a Markov chain generated by the Metropolis-Hastings algorithm. Assume $(X_n)_{n\in\mathbb N_0}$ is stationary and consider the ergodic averages $$A_n:=\frac1n\sum_{...
1
vote
3answers
97 views

Parameter estimation from single dynamic experiment. How many degrees of freedom?

I am estimating a parameter (diffusivity) from a set of experimental data coming from the same dynamic experiment: it can be summarised as changing the oxygen partial pressure around a solid and ...
1
vote
0answers
46 views

Convergence in distribution of $\sqrt{n-3}(z -\zeta)$

Let $(x_i,y_i)$ be iid $(x,y), i = 1,..,n$ be any bivariate sample. The correlation coefficient \begin{equation} \begin{split} \rho & = \text{cor}(x,y) \\ & = \frac{\text{cov}(x,y)}{\sqrt{\...
2
votes
1answer
35 views

Asymptotic Distribution of Covariance

I've seen a lot of questions revolving around the asymptotic distribution regarding the sample variance, such as $\sqrt{n} (s^2 - \sigma^2) \xrightarrow{d} N(0, \mu_4 - \sigma^4)$, however, what would ...
0
votes
1answer
54 views

Asymptotic Distribution of Minimum Uniform Random Variables

I've been working on this problem for a while, and I've made some progress, but I'm still stuck on some parts. I was hoping to get some assistance with this! Let $M_n = \min(X_1, ..., X_n)$ where $...
2
votes
1answer
42 views

What is the variance of the estimator in ordinary least squares with correlated residuals

If we assumed that $y \sim N(X\beta,S)$ where S= $\sigma^2\begin{bmatrix} 1 & \rho & \rho &...\\ \rho & 1 & \rho &...\\ \rho & \rho & 1 &...\\ \rho & \rho &...
0
votes
0answers
27 views

Comparing dimensionality

I am reading a couple of papers, The first one claims that $p_n = O(n^a)$ for some $0 \leq a < \infty$ The second paper claims $p_n = o(\exp(n^\varepsilon))$ for some $0< \varepsilon < .2$ ...
0
votes
0answers
86 views

Variance reduction of an estimator arising from the marginal destribution of a Metropolis-Hastings chain

Let $(E,\mathcal E,\lambda)$ and $(E',\mathcal E',\lambda')$ be measure spaces $f\in L^2(\lambda)$ $I$ be a finite nonempty set $\varphi_i:E'\to E$ be bijective $(\mathcal E',\mathcal E)$-measurable ...
0
votes
0answers
51 views

Maximal inequality for quadratic forms as functions of matrices?

In general, an $M$-estimator is defined as a maximizer of some objective function $Q_n(\theta)$: $\hat{\theta} = \arg\max_{\theta} Q_n(\theta)$. Suppose that $q(\theta) = E Q_n(\theta)$ is uniquely ...
0
votes
0answers
21 views

Fitting a regression model with spurious variables: do they vanish with large samples?

I am fitting the usual linear regression model $$y_j = x_j^T\beta + e_j,$$ where the errors $e_j$ are iid normal with unknown variance. If the vector of covariates $x_j$ contain spurious variables, ...
0
votes
1answer
29 views

How are Wald type tests better asymptotically? [closed]

I heard that Wald tests are better asymptotically than other tests. But what does this mean? Does it mean that it's better at testing for population level differences? Is it better for "big data" and ...
2
votes
0answers
24 views

Asymptotic approximation of log-probability using first four moments

Consider a random variable $X \sim p_{n,\theta}$ where the first four moments are given by known functions: $$\begin{matrix} \ \ \ \ \ \ \mathbb{E}(X) \equiv \mu(n,\theta) & & & \ \ \ \ \ ...
0
votes
0answers
34 views

Martingale Difference Sequence CLT

Could you provide me with the proof of the following: $$n^{-1/2} \cdot\sum_t a_{t-j}e_t$$ converges to normal distribution as $n$ goes infinity by martingale difference sequence CLT where $a_t = \...
3
votes
2answers
97 views

Asymptotic normality of random vector

If each component of a random vector is asymptotically normal by the central limit theorem (CLT), can anything be said about the asymptotic distribution of the random vector? The components are not ...
0
votes
0answers
21 views

Hausman test statistic - to multiply or not to multiply by n

I am having some serious doubts regarding the formula of the Hausman statistic for the case in which I compare OLS and IV estimates. I am getting confused with what my references are giving me. What ...
0
votes
0answers
45 views

Limiting Distribution of $n\left[Y_n\right]$ where $Y_n$ is the minimum of a sample of size n from Uniform$\left(0,\theta\right)$ distribution

Suppose $X_1,X_2,\dots,X_n$ is a random sample from Uniform$(0,\theta)$ for some unknown $\theta > 0$. Let $Y_n$ be the minimum of $X_1,X_2,\dots,X_n$. (a) Suppose $F_n$ is the CDF of $nY_n$. Show ...
0
votes
1answer
25 views

Why are oracle inequalities called that way?

The oracle property is an asymptotic property of an estimator, and is about variable selection: An estimator $\hat \beta_n$ satisfies the oracle property if in the limit of $n\to \infty$, the ...
2
votes
0answers
59 views

Asymptotic normality of quadratic form?

Let $X$ be a $p$-dimensional vector that is asymptotically normal such that $$\sqrt{n}(X - \mu_X) \stackrel{d}\longrightarrow N(0, \Sigma)$$, and let $H$ be a random $p\times p$ symmetric matrix, ...
1
vote
0answers
42 views

Distribution of variance of $X(X^TX)^{-1}X^TZ$?

I had a question that I couldn't quite wrap my mind around. Essentially, it's this. Suppose $x_i$ is a random $k$-vector, and $X$ is an $n\times k$ vector that is $n$ i.i.d. copies of this $x_i$ "...
1
vote
0answers
19 views

Counting the number of tuples $(i,i',k,k')$ satisfying some conditions

I'm struggling to find the cardinality of a set that will be presented below, as well as an upper bound for it. It is an engaging problem and perhaps not trivial. I will give now the baseline of the ...
1
vote
1answer
39 views

Generalizing Bayesian methods by assuming a “distribution of distributions” instead of a prior

Bayesian methods assume a prior distribution with several hyperparameters. Unfortunately, this is asymptotically incorrect, because distributions in the real world are never exact. For example, the ...
0
votes
0answers
28 views

Finding a test using asymptotic theory. for Poisson $(\lambda)$

If we have a sample of Poisson $(\lambda)$ (a) Find a test for $H_0: \lambda =2$ vs $ H_a: \lambda =\lambda_1> 2$ (b) Find a test using asymptotic theory. (c) Compare the results in en (a) y (...
2
votes
0answers
94 views

Asymptotic relationship between tests of compact composite nulls and one sided tests

A motivating example Let $x_t\sim N(\mu,1)$ for $t=1,\dots,T$. Consider testing the null $H_0: \mu\in[-1,1]$ against the alternative $H_1:\mu\in\mathbb{R}\setminus [-1,1]$. The natural (UMPU?) test ...
3
votes
0answers
46 views

Understanding simple LRT test asymptotic using Taylor expansion?

I am trying to understand the proof that the LRT test for $$H_0: \theta = \theta_0 \quad vs \quad H_1: \theta \neq \theta_0$$ is asymptotically chisquared distributed with one degree of freedom. I ...
0
votes
0answers
24 views

confidence interval for the variance with unknown distribution

Let $X_i \sim \text{iid}(\mu, \sigma^2)$ (the question does not specify whether or not $\mu$ and $\sigma^2$ are known). I have to show the confidence interval for the variance. Since $X_i$ is not ...