# How can I find the asymptotic relative efficiency of two quantities, estimating $\sigma$?

Let $$X_1,...,X_n$$ be a random sample from $$N(0,\sigma^2)$$, where $$\sigma>0$$ is unknown. We try to estimate $$\sigma$$ using $$T_1=\sqrt{\frac{\pi}{2}}\frac{1}{n}\sum^n_{i=1}|X_i|$$ and $$T_2=\sqrt{\frac{1}{n}\sum^n_{i=1}(X_i^2)}$$. What is the asymptotic relative efficiency of $$T_1$$ w.r.t $$T_2$$?

my work:

If these two estimators are such that $$T_1 \sim AN(\tau(\theta),\sigma^2_1/n)$$ and $$T_2 \sim AN(\tau(\theta),\sigma^2_2/n)$$, then $$ARE(T_1,T_2)=\sigma^2_2/\sigma^2_1$$.

I am having troubles finding the asymptotic distributions of $$T_1,T_2$$. I believe that I will need to use the Delta Method and properties of the half-normal distribution, since $$T_1$$ makes use of the random variable $$|X|\sim HN(\sigma)$$.

I have made some slight progress in finding the asymptotic distribution of $$T_2$$. By CLT, $$\frac{1}{n}\sum(X_i^2)\sim AN(\sigma^2,Var(X^2)/n)$$. Then, by letting $$g(z)=\sqrt{z}$$, Delta Method yields $$g(\frac{1}{n}\sum(X_i^2))=T_2 \sim AN(\sigma, \frac{Var(X^2)}{n}\cdot\frac{1}{4\sigma^2})$$. However, I do not know how to evaluate $$Var(X^2)$$.

I used a similar approach for finding the asymptotic distribution of $$T_1$$. Since $$|X| \sim HN(\sigma)$$, we have $$\bar{|X|} \sim AN(\frac{\sigma\sqrt{2}}{\sqrt{\pi}},\frac{\sigma^2(1-2/\pi)}{n})$$ from CLT. Then, we use Delta Method with $$g(z)=z\sqrt{\pi/2}$$ to get $$T_1 \sim AN(\sigma, \frac{\sigma^2\pi(1-2/\pi)}{2n})$$. How can I complete this problem?

Your method works fine. Moments of normal distribution are well known. In particular the fourth moment $$X_1$$ is $$3\sigma^4$$, which can be shown from definition or moment generating functions for instance. I remember it from well-known recurrence relations of the moments. This gives $$\operatorname{Var}(X_1^2)=E(X_1^4)-(E(X_1^2))^2=2\sigma^4$$

Alternatively you can note that $$\frac{X_1^2}{\sigma^2}\sim \chi^2_1\implies \frac1{\sigma^4}\operatorname{Var}(X_1^2)=2$$.

Indeed by CLT,

$$\sqrt n\left(\frac1n\sum_{i=1}^n |X_i|-\sigma\right)\stackrel{d}\longrightarrow N\left(0,\sigma^2\left(1-\frac2{\pi}\right)\right)$$

So variance of $$T_1$$ for large $$n$$ is $$\operatorname{Var}(T_1)\approx \frac{\pi}2\cdot\frac{\sigma^2}{n}\left(1-\frac2{\pi}\right)$$

Again, CLT gives

$$\sqrt n\left(\frac1n\sum_{i=1}^n X_i^2-\sigma^2\right)\stackrel{d}\longrightarrow N\left(0,2\sigma^4\right)$$

By Delta-method, this implies

$$\sqrt n\left(T_2-\sigma\right)\stackrel{d}\longrightarrow N\left(0,\frac{\sigma^2}2\right)$$

That is, variance of $$T_2$$ for large $$n$$ is $$\operatorname{Var}(T_2)\approx \frac{\sigma^2}{2n}$$

Both $$T_1$$ and $$T_2$$ are asymptotically unbiased for $$\sigma$$ (with $$T_1$$ being exactly unbiased).

So relative efficiency of $$T_1$$ compared to $$T_2$$ for large $$n$$ is $$\frac{\operatorname{Var}(T_2)}{\operatorname{Var}(T_1)}\approx \frac1{\pi-2}$$

• This is perfect. I find your solution very elegant. I like how you transformed $X_1^2$ to a Chi-squared random variable to solve for $Var(X^2)$. I haven't really studied the moments of the normal distribution past the first two. I will give more mind to them as I proceed in my studies. Thanks for the answer! – Ron Snow Apr 18 '20 at 20:45

For the first estimator, you are correct that you have a half-normal distribution for the individual pieces. Since $$|X_1|,...,|X_n| \sim \text{IID HalfN}(\sigma)$$ it can easily be shown that this estimator is unbiased. Its variance is given by:

\begin{aligned} \mathbb{V}(T_1) &= \mathbb{E}(T_1^2) - \sigma^2 \\[6pt] &= \frac{\pi}{2} \cdot \frac{1}{n^2} \cdot \mathbb{E} \Bigg( \bigg( \sum_{i=1}^n |X_i| \bigg)^2 \Bigg) - \sigma^2 \\[6pt] &= \frac{\pi}{2} \cdot \frac{1}{n^2} \cdot \mathbb{E} \Bigg( \sum_{i=1}^n \sum_{j=1}^n |X_i| |X_j| \Bigg) - \sigma^2 \\[6pt] &= \frac{\pi}{2} \cdot \frac{1}{n^2} \cdot \Bigg( \sum_{i=1}^n \mathbb{E} (X_i^2) + \sum_{i \neq j} \mathbb{E} |X_i| \mathbb{E} |X_j| \Bigg) - \sigma^2 \\[6pt] &= \frac{\pi}{2} \cdot \frac{1}{n^2} \cdot \Bigg( \sum_{i=1}^n \sigma^2 + \sum_{i \neq j} \frac{2}{\pi} \sigma^2 \Bigg) - \sigma^2 \\[6pt] &= \frac{\pi}{2} \cdot \frac{1}{n^2} \cdot \Bigg( n \sigma^2 + n(n-1) \frac{2}{\pi} \sigma^2 \Bigg) - \sigma^2 \\[6pt] &= \frac{\pi}{2} \cdot \frac{\sigma^2}{n} \cdot \Bigg( 1 + (n-1) \frac{2}{\pi} \Bigg) - \sigma^2 \\[6pt] &= \bigg( \frac{\pi}{2n} + \frac{n-1}{n} - 1 \bigg) \sigma^2 \\[6pt] &= \frac{\pi-2}{2} \cdot \frac{\sigma^2}{n}. \\[6pt] \end{aligned}

The second estimator follows a scaled chi distribution given by $$T_2 \sim (\sigma/\sqrt{n}) \cdot \text{Chi}(n)$$. Its exact variance contains a term that is the ratio of gamma functions, which we will denote as:

$$H(n) \equiv \frac{\Gamma(\tfrac{n+1}{2})}{\Gamma(\tfrac{n}{2})}.$$

Using the asymptotic expansion of the ratio of gamma functions in Tricomi and Erdelyi (1951), we obtain the result:

$$H(n)^2 = \frac{n}{2} - \frac{1}{4} + \mathcal{O}(n^{-1}).$$

Using this function, and the formula for the variance of the chi distribution, the variance of the second estimator is given by:

\begin{aligned} \mathbb{V}(T_2) &= \frac{\sigma^2}{n} \cdot ( n - 2 H(n)^2 ) \\[6pt] &= \frac{\sigma^2}{n} \cdot \bigg( \frac{1}{2} + \mathcal{O}(n^{-1}) \bigg) \\[6pt] \end{aligned}

Thus, the asymptotic relative efficiency of the estimators is:

$$\lim_{n \rightarrow \infty} \frac{\mathbb{V}(T_1)}{\mathbb{V}(T_2)} = \lim_{n \rightarrow \infty} \frac{\pi - 2}{2n - 4 H(n)^2} = \lim_{n \rightarrow \infty} \frac{\pi - 2}{1 + \mathcal{O}(n^{-1})} = \pi - 2.$$

Since this value is greater than one, we see here that the first estimator is less efficient than the second estimator as $$n \rightarrow \infty$$.

• Although you are bringing up material that I haven't seen yet, I absolutely love your use of the scaled chi-square distribution. Also, +1 for referencing that paper and your detailed work. – Ron Snow Apr 18 '20 at 20:47