Variance of an Unbiased Estimator for $\sigma^2$

Question

Let $X_1, X_2,...X_n\sim N(0,\sigma^2)$ independently. Define $$Q=\frac{1}{2(n-1)}\sum_{i=1}^{n-1}(X_{i+1}-X_i)^2$$ I already proved that this Q is an unbiased estimator of $\sigma^2$. Now I'm stuck with calculating its variance, I've tried using Chi-square but then I realized these $(X_{i+1}-X_i)$ are not independent. Can you guys help me with this? Many thanks in advance.

Answer 1

The easiest way to do this problem is by using vector algebra, re-expressing the estimator as a quadratic form in vector notation:

$$Q = \frac{1}{2(n-1)} \mathbf{X}^\text{T} \mathbf{\Delta} \mathbf{X} \quad \quad \mathbf{\Delta} \equiv \begin{bmatrix} 1 & -1 & 0 & 0 & \cdots & 0 & 0 & 0 & 0 \\ -1 & 2 & -1 & 0 & \cdots & 0 & 0 & 0 & 0 \\ 0 & -1 & 2 & -1 & \cdots & 0 & 0 & 0 & 0 \\ 0 & 0 & -1 & 2 & \cdots & 0 & 0 & 0 & 0 \\ \vdots & \vdots & \vdots & \vdots & \ddots & \vdots & \vdots & \vdots \\ 0 & 0 & 0 & 0 & \cdots & 2 & -1 & 0 & 0 \\ 0 & 0 & 0 & 0 & \cdots & -1 & 2 & -1 & 0 \\ 0 & 0 & 0 & 0 & \cdots & 0 & -1 & 2 & -1 \\ 0 & 0 & 0 & 0 & \cdots & 0 & 0 & -1 & 1 \\ \end{bmatrix}.$$

Since $\mathbf{X} \sim \text{N}(\boldsymbol{0},\sigma^2 \boldsymbol{I})$ we can compute the expected value of the quadratic form to confirm that the estimator is unbiased:

$$\begin{equation} \begin{aligned} \mathbb{E}(Q) &= \frac{1}{2(n-1)} \cdot \mathbb{E}(\mathbf{X}^\text{T} \mathbf{\Delta} \mathbf{X}) \\[6pt] &= \frac{\sigma^2}{2(n-1)} \cdot \text{tr}(\mathbf{\Delta} \boldsymbol{I}) \\[6pt] &= \frac{\sigma^2}{2(n-1)} \cdot (1 + 2 + 2 + \cdots + 2 + 2 + 1) \\[6pt] &= \frac{\sigma^2}{2(n-1)} \cdot (2n-2) \\[6pt] &= \frac{2(n-1)}{2(n-1)} \cdot \sigma^2 = \sigma^2. \\[6pt] \end{aligned} \end{equation}$$

This confirms that the estimator is unbiased. Now, to get the variance of the estimator we can compute the variance of a quadratic form for the case of a joint normal random vector, which gives:

$$\begin{equation} \begin{aligned} \mathbb{V}(Q) &= \frac{1}{4(n-1)^2} \cdot \mathbb{V}(\mathbf{X}^\text{T} \mathbf{\Delta} \mathbf{X}) \\[6pt] &= \frac{\sigma^4}{2(n-1)^2} \cdot \text{tr}(\mathbf{\Delta} \boldsymbol{I} \mathbf{\Delta} \boldsymbol{I}) \\[6pt] &= \frac{\sigma^4}{2(n-1)^2} \cdot \text{tr}(\mathbf{\Delta}^2) \\[6pt] &= \frac{\sigma^4}{2(n-1)^2} \cdot (2 + 6 + 6 + \cdots + 6 + 6 + 2) \\[6pt] &= \frac{\sigma^4}{2(n-1)^2} \cdot (6n - 8) \\[6pt] &= \frac{3n-4}{(n-1)^2} \cdot \sigma^4 \\[6pt] &= \frac{3n-4}{2n-2} \cdot \frac{2}{n-1} \cdot \sigma^4. \\[6pt] \end{aligned} \end{equation}$$

This gives us an expression for the variance of the estimator. (I have framed it in this form to compare it to an alternative estimator below.) As $n \rightarrow \infty$ we have $\mathbb{V}(Q) \rightarrow 0$ so it is a consistent estimator. It is worth contrasting the variance of this estimator with the variance of the sample variance estimator (see e.g., O'Neill 2014, Result 3), which is:

$$\mathbb{V}(S^2) = \frac{2}{n-1} \cdot \sigma^4.$$

Comparing these results we see that the estimators have the same variance when $n=2$, and when $n>2$ the sample variance estimator has lower variance than the present estimator. In other words, the sample variance is a more efficient estimator than the present estimator.