# Variance of an Unbiased Estimator for $\sigma^2$

Let $$X_1, X_2,...X_n\sim N(0,\sigma^2)$$ independently. Define $$Q=\frac{1}{2(n-1)}\sum_{i=1}^{n-1}(X_{i+1}-X_i)^2$$ I already proved that this Q is an unbiased estimator of $$\sigma^2$$. Now I'm stuck with calculating its variance, I've tried using Chi-square but then I realized these $$(X_{i+1}-X_i)$$ are not independent. Can you guys help me with this? Many thanks in advance.

## 1 Answer

The easiest way to do this problem is by using vector algebra, re-expressing the estimator as a quadratic form in vector notation:

$$Q = \frac{1}{2(n-1)} \mathbf{X}^\text{T} \mathbf{\Delta} \mathbf{X} \quad \quad \mathbf{\Delta} \equiv \begin{bmatrix} 1 & -1 & 0 & 0 & \cdots & 0 & 0 & 0 & 0 \\ -1 & 2 & -1 & 0 & \cdots & 0 & 0 & 0 & 0 \\ 0 & -1 & 2 & -1 & \cdots & 0 & 0 & 0 & 0 \\ 0 & 0 & -1 & 2 & \cdots & 0 & 0 & 0 & 0 \\ \vdots & \vdots & \vdots & \vdots & \ddots & \vdots & \vdots & \vdots \\ 0 & 0 & 0 & 0 & \cdots & 2 & -1 & 0 & 0 \\ 0 & 0 & 0 & 0 & \cdots & -1 & 2 & -1 & 0 \\ 0 & 0 & 0 & 0 & \cdots & 0 & -1 & 2 & -1 \\ 0 & 0 & 0 & 0 & \cdots & 0 & 0 & -1 & 1 \\ \end{bmatrix}.$$

Since $$\mathbf{X} \sim \text{N}(\boldsymbol{0},\sigma^2 \boldsymbol{I})$$ we can compute the expected value of the quadratic form to confirm that the estimator is unbiased:

\begin{equation} \begin{aligned} \mathbb{E}(Q) &= \frac{1}{2(n-1)} \cdot \mathbb{E}(\mathbf{X}^\text{T} \mathbf{\Delta} \mathbf{X}) \\[6pt] &= \frac{\sigma^2}{2(n-1)} \cdot \text{tr}(\mathbf{\Delta} \boldsymbol{I}) \\[6pt] &= \frac{\sigma^2}{2(n-1)} \cdot (1 + 2 + 2 + \cdots + 2 + 2 + 1) \\[6pt] &= \frac{\sigma^2}{2(n-1)} \cdot (2n-2) \\[6pt] &= \frac{2(n-1)}{2(n-1)} \cdot \sigma^2 = \sigma^2. \\[6pt] \end{aligned} \end{equation}

This confirms that the estimator is unbiased. Now, to get the variance of the estimator we can compute the variance of a quadratic form for the case of a joint normal random vector, which gives:

\begin{equation} \begin{aligned} \mathbb{V}(Q) &= \frac{1}{4(n-1)^2} \cdot \mathbb{V}(\mathbf{X}^\text{T} \mathbf{\Delta} \mathbf{X}) \\[6pt] &= \frac{\sigma^4}{2(n-1)^2} \cdot \text{tr}(\mathbf{\Delta} \boldsymbol{I} \mathbf{\Delta} \boldsymbol{I}) \\[6pt] &= \frac{\sigma^4}{2(n-1)^2} \cdot \text{tr}(\mathbf{\Delta}^2) \\[6pt] &= \frac{\sigma^4}{2(n-1)^2} \cdot (2 + 6 + 6 + \cdots + 6 + 6 + 2) \\[6pt] &= \frac{\sigma^4}{2(n-1)^2} \cdot (6n - 8) \\[6pt] &= \frac{3n-4}{(n-1)^2} \cdot \sigma^4 \\[6pt] &= \frac{3n-4}{2n-2} \cdot \frac{2}{n-1} \cdot \sigma^4. \\[6pt] \end{aligned} \end{equation}

This gives us an expression for the variance of the estimator. (I have framed it in this form to compare it to an alternative estimator below.) As $$n \rightarrow \infty$$ we have $$\mathbb{V}(Q) \rightarrow 0$$ so it is a consistent estimator. It is worth contrasting the variance of this estimator with the variance of the sample variance estimator (see e.g., O'Neill 2014, Result 3), which is:

$$\mathbb{V}(S^2) = \frac{2}{n-1} \cdot \sigma^4.$$

Comparing these results we see that the estimators have the same variance when $$n=2$$, and when $$n>2$$ the sample variance estimator has lower variance than the present estimator. In other words, the sample variance is a more efficient estimator than the present estimator.

• Thank you Ben, I had a look at your previous answer and tried to proceed but apparently the expectation of cross products was too tedious. I like this matrix approach. – diidoobiib Feb 5 '19 at 3:32
• I have 1 question, the combination of univariate random normal variables is not necessarily multivariate normal. How can we make sure $\textbf{X}$ is multivariate normal? – diidoobiib Feb 5 '19 at 3:35
• I am assuming in your initial specification of $X_1,...,X_n$ that they are jointly independent. If that is correct, then combined with the marginal normal distribution, they are jointly normal. – Ben Feb 5 '19 at 3:42
• Thanks a lot. I've learned new things from your answer, especially the variance of quadratic form. – diidoobiib Feb 5 '19 at 3:53