Let $X_1, X_2,...X_n\sim N(0,\sigma^2)$ independently. Define $$Q=\frac{1}{2(n-1)}\sum_{i=1}^{n-1}(X_{i+1}-X_i)^2$$ I already proved that this Q is an unbiased estimator of $\sigma^2$. Now I'm stuck with calculating its variance, I've tried using Chi-square but then I realized these $(X_{i+1}-X_i)$ are not independent. Can you guys help me with this? Many thanks in advance.
The easiest way to do this problem is by using vector algebra, re-expressing the estimator as a quadratic form in vector notation:
$$Q = \frac{1}{2(n-1)} \mathbf{X}^\text{T} \mathbf{\Delta} \mathbf{X} \quad \quad \mathbf{\Delta} \equiv \begin{bmatrix} 1 & -1 & 0 & 0 & \cdots & 0 & 0 & 0 & 0 \\ -1 & 2 & -1 & 0 & \cdots & 0 & 0 & 0 & 0 \\ 0 & -1 & 2 & -1 & \cdots & 0 & 0 & 0 & 0 \\ 0 & 0 & -1 & 2 & \cdots & 0 & 0 & 0 & 0 \\ \vdots & \vdots & \vdots & \vdots & \ddots & \vdots & \vdots & \vdots \\ 0 & 0 & 0 & 0 & \cdots & 2 & -1 & 0 & 0 \\ 0 & 0 & 0 & 0 & \cdots & -1 & 2 & -1 & 0 \\ 0 & 0 & 0 & 0 & \cdots & 0 & -1 & 2 & -1 \\ 0 & 0 & 0 & 0 & \cdots & 0 & 0 & -1 & 1 \\ \end{bmatrix}.$$
Since $\mathbf{X} \sim \text{N}(\boldsymbol{0},\sigma^2 \boldsymbol{I})$ we can compute the expected value of the quadratic form to confirm that the estimator is unbiased:
$$\begin{equation} \begin{aligned} \mathbb{E}(Q) &= \frac{1}{2(n-1)} \cdot \mathbb{E}(\mathbf{X}^\text{T} \mathbf{\Delta} \mathbf{X}) \\[6pt] &= \frac{\sigma^2}{2(n-1)} \cdot \text{tr}(\mathbf{\Delta} \boldsymbol{I}) \\[6pt] &= \frac{\sigma^2}{2(n-1)} \cdot (1 + 2 + 2 + \cdots + 2 + 2 + 1) \\[6pt] &= \frac{\sigma^2}{2(n-1)} \cdot (2n-2) \\[6pt] &= \frac{2(n-1)}{2(n-1)} \cdot \sigma^2 = \sigma^2. \\[6pt] \end{aligned} \end{equation}$$
This confirms that the estimator is unbiased. Now, to get the variance of the estimator we can compute the variance of a quadratic form for the case of a joint normal random vector, which gives:
$$\begin{equation} \begin{aligned} \mathbb{V}(Q) &= \frac{1}{4(n-1)^2} \cdot \mathbb{V}(\mathbf{X}^\text{T} \mathbf{\Delta} \mathbf{X}) \\[6pt] &= \frac{\sigma^4}{2(n-1)^2} \cdot \text{tr}(\mathbf{\Delta} \boldsymbol{I} \mathbf{\Delta} \boldsymbol{I}) \\[6pt] &= \frac{\sigma^4}{2(n-1)^2} \cdot \text{tr}(\mathbf{\Delta}^2) \\[6pt] &= \frac{\sigma^4}{2(n-1)^2} \cdot (2 + 6 + 6 + \cdots + 6 + 6 + 2) \\[6pt] &= \frac{\sigma^4}{2(n-1)^2} \cdot (6n - 8) \\[6pt] &= \frac{3n-4}{(n-1)^2} \cdot \sigma^4 \\[6pt] &= \frac{3n-4}{2n-2} \cdot \frac{2}{n-1} \cdot \sigma^4. \\[6pt] \end{aligned} \end{equation}$$
This gives us an expression for the variance of the estimator. (I have framed it in this form to compare it to an alternative estimator below.) As $n \rightarrow \infty$ we have $\mathbb{V}(Q) \rightarrow 0$ so it is a consistent estimator. It is worth contrasting the variance of this estimator with the variance of the sample variance estimator (see e.g., O'Neill 2014, Result 3), which is:
$$\mathbb{V}(S^2) = \frac{2}{n-1} \cdot \sigma^4.$$
Comparing these results we see that the estimators have the same variance when $n=2$, and when $n>2$ the sample variance estimator has lower variance than the present estimator. In other words, the sample variance is a more efficient estimator than the present estimator.
-
$\begingroup$ Thank you Ben, I had a look at your previous answer and tried to proceed but apparently the expectation of cross products was too tedious. I like this matrix approach. $\endgroup$ – diidoobiib Feb 5 '19 at 3:32
-
$\begingroup$ I have 1 question, the combination of univariate random normal variables is not necessarily multivariate normal. How can we make sure $\textbf{X}$ is multivariate normal? $\endgroup$ – diidoobiib Feb 5 '19 at 3:35
-
$\begingroup$ I am assuming in your initial specification of $X_1,...,X_n$ that they are jointly independent. If that is correct, then combined with the marginal normal distribution, they are jointly normal. $\endgroup$ – Ben Feb 5 '19 at 3:42
-
1$\begingroup$ Thanks a lot. I've learned new things from your answer, especially the variance of quadratic form. $\endgroup$ – diidoobiib Feb 5 '19 at 3:53