2
$\begingroup$

Let $X_1,...,X_n$ be a random sample from $N(0,\sigma^2)$, where $\sigma>0$ is unknown. We try to estimate $\sigma$ using $T_1=\sqrt{\frac{\pi}{2}}\frac{1}{n}\sum^n_{i=1}|X_i|$ and $T_2=\sqrt{\frac{1}{n}\sum^n_{i=1}(X_i^2)}$. What is the asymptotic relative efficiency of $T_1$ w.r.t $T_2$?

my work:

If these two estimators are such that $T_1 \sim AN(\tau(\theta),\sigma^2_1/n)$ and $T_2 \sim AN(\tau(\theta),\sigma^2_2/n)$, then $ARE(T_1,T_2)=\sigma^2_2/\sigma^2_1$.

I am having troubles finding the asymptotic distributions of $T_1,T_2$. I believe that I will need to use the Delta Method and properties of the half-normal distribution, since $T_1$ makes use of the random variable $|X|\sim HN(\sigma)$.

I have made some slight progress in finding the asymptotic distribution of $T_2$. By CLT, $\frac{1}{n}\sum(X_i^2)\sim AN(\sigma^2,Var(X^2)/n)$. Then, by letting $g(z)=\sqrt{z}$, Delta Method yields $g(\frac{1}{n}\sum(X_i^2))=T_2 \sim AN(\sigma, \frac{Var(X^2)}{n}\cdot\frac{1}{4\sigma^2})$. However, I do not know how to evaluate $Var(X^2)$.

I used a similar approach for finding the asymptotic distribution of $T_1$. Since $|X| \sim HN(\sigma)$, we have $\bar{|X|} \sim AN(\frac{\sigma\sqrt{2}}{\sqrt{\pi}},\frac{\sigma^2(1-2/\pi)}{n})$ from CLT. Then, we use Delta Method with $g(z)=z\sqrt{\pi/2}$ to get $T_1 \sim AN(\sigma, \frac{\sigma^2\pi(1-2/\pi)}{2n})$. How can I complete this problem?

$\endgroup$
1
$\begingroup$

Your method works fine. Moments of normal distribution are well known. In particular the fourth moment $X_1$ is $3\sigma^4$, which can be shown from definition or moment generating functions for instance. I remember it from well-known recurrence relations of the moments. This gives $$\operatorname{Var}(X_1^2)=E(X_1^4)-(E(X_1^2))^2=2\sigma^4$$

Alternatively you can note that $\frac{X_1^2}{\sigma^2}\sim \chi^2_1\implies \frac1{\sigma^4}\operatorname{Var}(X_1^2)=2$.

Indeed by CLT,

$$\sqrt n\left(\frac1n\sum_{i=1}^n |X_i|-\sigma\right)\stackrel{d}\longrightarrow N\left(0,\sigma^2\left(1-\frac2{\pi}\right)\right)$$

So variance of $T_1$ for large $n$ is $$\operatorname{Var}(T_1)\approx \frac{\pi}2\cdot\frac{\sigma^2}{n}\left(1-\frac2{\pi}\right)$$

Again, CLT gives

$$\sqrt n\left(\frac1n\sum_{i=1}^n X_i^2-\sigma^2\right)\stackrel{d}\longrightarrow N\left(0,2\sigma^4\right)$$

By Delta-method, this implies

$$\sqrt n\left(T_2-\sigma\right)\stackrel{d}\longrightarrow N\left(0,\frac{\sigma^2}2\right)$$

That is, variance of $T_2$ for large $n$ is $$\operatorname{Var}(T_2)\approx \frac{\sigma^2}{2n}$$

Both $T_1$ and $T_2$ are asymptotically unbiased for $\sigma$ (with $T_1$ being exactly unbiased).

So relative efficiency of $T_1$ compared to $T_2$ for large $n$ is $$\frac{\operatorname{Var}(T_2)}{\operatorname{Var}(T_1)}\approx \frac1{\pi-2}$$

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ This is perfect. I find your solution very elegant. I like how you transformed $X_1^2$ to a Chi-squared random variable to solve for $Var(X^2)$. I haven't really studied the moments of the normal distribution past the first two. I will give more mind to them as I proceed in my studies. Thanks for the answer! $\endgroup$ – Ron Snow Apr 18 at 20:45
2
$\begingroup$

For the first estimator, you are correct that you have a half-normal distribution for the individual pieces. Since $|X_1|,...,|X_n| \sim \text{IID HalfN}(\sigma)$ it can easily be shown that this estimator is unbiased. Its variance is given by:

$$\begin{aligned} \mathbb{V}(T_1) &= \mathbb{E}(T_1^2) - \sigma^2 \\[6pt] &= \frac{\pi}{2} \cdot \frac{1}{n^2} \cdot \mathbb{E} \Bigg( \bigg( \sum_{i=1}^n |X_i| \bigg)^2 \Bigg) - \sigma^2 \\[6pt] &= \frac{\pi}{2} \cdot \frac{1}{n^2} \cdot \mathbb{E} \Bigg( \sum_{i=1}^n \sum_{j=1}^n |X_i| |X_j| \Bigg) - \sigma^2 \\[6pt] &= \frac{\pi}{2} \cdot \frac{1}{n^2} \cdot \Bigg( \sum_{i=1}^n \mathbb{E} (X_i^2) + \sum_{i \neq j} \mathbb{E} |X_i| \mathbb{E} |X_j| \Bigg) - \sigma^2 \\[6pt] &= \frac{\pi}{2} \cdot \frac{1}{n^2} \cdot \Bigg( \sum_{i=1}^n \sigma^2 + \sum_{i \neq j} \frac{2}{\pi} \sigma^2 \Bigg) - \sigma^2 \\[6pt] &= \frac{\pi}{2} \cdot \frac{1}{n^2} \cdot \Bigg( n \sigma^2 + n(n-1) \frac{2}{\pi} \sigma^2 \Bigg) - \sigma^2 \\[6pt] &= \frac{\pi}{2} \cdot \frac{\sigma^2}{n} \cdot \Bigg( 1 + (n-1) \frac{2}{\pi} \Bigg) - \sigma^2 \\[6pt] &= \bigg( \frac{\pi}{2n} + \frac{n-1}{n} - 1 \bigg) \sigma^2 \\[6pt] &= \frac{\pi-2}{2} \cdot \frac{\sigma^2}{n}. \\[6pt] \end{aligned}$$

The second estimator follows a scaled chi distribution given by $T_2 \sim (\sigma/\sqrt{n}) \cdot \text{Chi}(n)$. Its exact variance contains a term that is the ratio of gamma functions, which we will denote as:

$$H(n) \equiv \frac{\Gamma(\tfrac{n+1}{2})}{\Gamma(\tfrac{n}{2})}.$$

Using the asymptotic expansion of the ratio of gamma functions in Tricomi and Erdelyi (1951), we obtain the result:

$$H(n)^2 = \frac{n}{2} - \frac{1}{4} + \mathcal{O}(n^{-1}).$$

Using this function, and the formula for the variance of the chi distribution, the variance of the second estimator is given by:

$$\begin{aligned} \mathbb{V}(T_2) &= \frac{\sigma^2}{n} \cdot ( n - 2 H(n)^2 ) \\[6pt] &= \frac{\sigma^2}{n} \cdot \bigg( \frac{1}{2} + \mathcal{O}(n^{-1}) \bigg) \\[6pt] \end{aligned}$$

Thus, the asymptotic relative efficiency of the estimators is:

$$\lim_{n \rightarrow \infty} \frac{\mathbb{V}(T_1)}{\mathbb{V}(T_2)} = \lim_{n \rightarrow \infty} \frac{\pi - 2}{2n - 4 H(n)^2} = \lim_{n \rightarrow \infty} \frac{\pi - 2}{1 + \mathcal{O}(n^{-1})} = \pi - 2.$$

Since this value is greater than one, we see here that the first estimator is less efficient than the second estimator as $n \rightarrow \infty$.

| cite | improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ Although you are bringing up material that I haven't seen yet, I absolutely love your use of the scaled chi-square distribution. Also, +1 for referencing that paper and your detailed work. $\endgroup$ – Ron Snow Apr 18 at 20:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.