3
$\begingroup$

Let $X_1, X_2,...X_n\sim N(0,\sigma^2)$ independently. Define $$Q=\frac{1}{2(n-1)}\sum_{i=1}^{n-1}(X_{i+1}-X_i)^2$$ I already proved that this Q is an unbiased estimator of $\sigma^2$. Now I'm stuck with calculating its variance, I've tried using Chi-square but then I realized these $(X_{i+1}-X_i)$ are not independent. Can you guys help me with this? Many thanks in advance.

$\endgroup$
4
$\begingroup$

The easiest way to do this problem is by using vector algebra, re-expressing the estimator as a quadratic form in vector notation:

$$Q = \frac{1}{2(n-1)} \mathbf{X}^\text{T} \mathbf{\Delta} \mathbf{X} \quad \quad \mathbf{\Delta} \equiv \begin{bmatrix} 1 & -1 & 0 & 0 & \cdots & 0 & 0 & 0 & 0 \\ -1 & 2 & -1 & 0 & \cdots & 0 & 0 & 0 & 0 \\ 0 & -1 & 2 & -1 & \cdots & 0 & 0 & 0 & 0 \\ 0 & 0 & -1 & 2 & \cdots & 0 & 0 & 0 & 0 \\ \vdots & \vdots & \vdots & \vdots & \ddots & \vdots & \vdots & \vdots \\ 0 & 0 & 0 & 0 & \cdots & 2 & -1 & 0 & 0 \\ 0 & 0 & 0 & 0 & \cdots & -1 & 2 & -1 & 0 \\ 0 & 0 & 0 & 0 & \cdots & 0 & -1 & 2 & -1 \\ 0 & 0 & 0 & 0 & \cdots & 0 & 0 & -1 & 1 \\ \end{bmatrix}.$$

Since $\mathbf{X} \sim \text{N}(\boldsymbol{0},\sigma^2 \boldsymbol{I})$ we can compute the expected value of the quadratic form to confirm that the estimator is unbiased:

$$\begin{equation} \begin{aligned} \mathbb{E}(Q) &= \frac{1}{2(n-1)} \cdot \mathbb{E}(\mathbf{X}^\text{T} \mathbf{\Delta} \mathbf{X}) \\[6pt] &= \frac{\sigma^2}{2(n-1)} \cdot \text{tr}(\mathbf{\Delta} \boldsymbol{I}) \\[6pt] &= \frac{\sigma^2}{2(n-1)} \cdot (1 + 2 + 2 + \cdots + 2 + 2 + 1) \\[6pt] &= \frac{\sigma^2}{2(n-1)} \cdot (2n-2) \\[6pt] &= \frac{2(n-1)}{2(n-1)} \cdot \sigma^2 = \sigma^2. \\[6pt] \end{aligned} \end{equation}$$

This confirms that the estimator is unbiased. Now, to get the variance of the estimator we can compute the variance of a quadratic form for the case of a joint normal random vector, which gives:

$$\begin{equation} \begin{aligned} \mathbb{V}(Q) &= \frac{1}{4(n-1)^2} \cdot \mathbb{V}(\mathbf{X}^\text{T} \mathbf{\Delta} \mathbf{X}) \\[6pt] &= \frac{\sigma^4}{2(n-1)^2} \cdot \text{tr}(\mathbf{\Delta} \boldsymbol{I} \mathbf{\Delta} \boldsymbol{I}) \\[6pt] &= \frac{\sigma^4}{2(n-1)^2} \cdot \text{tr}(\mathbf{\Delta}^2) \\[6pt] &= \frac{\sigma^4}{2(n-1)^2} \cdot (2 + 6 + 6 + \cdots + 6 + 6 + 2) \\[6pt] &= \frac{\sigma^4}{2(n-1)^2} \cdot (6n - 8) \\[6pt] &= \frac{3n-4}{(n-1)^2} \cdot \sigma^4 \\[6pt] &= \frac{3n-4}{2n-2} \cdot \frac{2}{n-1} \cdot \sigma^4. \\[6pt] \end{aligned} \end{equation}$$

This gives us an expression for the variance of the estimator. (I have framed it in this form to compare it to an alternative estimator below.) As $n \rightarrow \infty$ we have $\mathbb{V}(Q) \rightarrow 0$ so it is a consistent estimator. It is worth contrasting the variance of this estimator with the variance of the sample variance estimator (see e.g., O'Neill 2014, Result 3), which is:

$$\mathbb{V}(S^2) = \frac{2}{n-1} \cdot \sigma^4.$$

Comparing these results we see that the estimators have the same variance when $n=2$, and when $n>2$ the sample variance estimator has lower variance than the present estimator. In other words, the sample variance is a more efficient estimator than the present estimator.

$\endgroup$
  • $\begingroup$ Thank you Ben, I had a look at your previous answer and tried to proceed but apparently the expectation of cross products was too tedious. I like this matrix approach. $\endgroup$ – diidoobiib Feb 5 at 3:32
  • $\begingroup$ I have 1 question, the combination of univariate random normal variables is not necessarily multivariate normal. How can we make sure $\textbf{X}$ is multivariate normal? $\endgroup$ – diidoobiib Feb 5 at 3:35
  • $\begingroup$ I am assuming in your initial specification of $X_1,...,X_n$ that they are jointly independent. If that is correct, then combined with the marginal normal distribution, they are jointly normal. $\endgroup$ – Ben Feb 5 at 3:42
  • 1
    $\begingroup$ Thanks a lot. I've learned new things from your answer, especially the variance of quadratic form. $\endgroup$ – diidoobiib Feb 5 at 3:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.