# Estimating the white noise variance

This is a mock-exam question:

The following measurements have been made

$$\hat{\gamma}(0) = 570, \quad \hat{\gamma}(1) = 505, \quad \hat{\gamma}(2) = 460, \quad \hat{\gamma}(3) = 420$$

Suppose that the data was generated from an $$AR(2)$$ model,

$$X_t - \phi_1 X_{t-1} - \phi_2 X_{t-2} = Z_t$$

where $$\{Z_t\} \sim WN(0,\sigma^2)$$. Estimate the variance $$\sigma^2$$.

What i've tried is simply to use the Yule-Walker equations to estimate the variance, $$\begin{equation*} \gamma(k) - \phi_1 \gamma(k - 1) - ... - \phi_p \gamma(k - p) = \begin{cases} 0, \quad &\text{if } k = 1, ..., p \\ \sigma^2, \quad &\text{if } k = 0 \end{cases} \end{equation*}$$ which, if $$k = 0$$, should give $$\gamma(0) = \hat \gamma(0) = \sigma^2 = 570$$. But the answer is $$\sigma^2 = 121$$.

What am I missing here?

$$\gamma(0)$$ is the variance of $$X_t$$, it is not in general equal to the variance of $$Z_t$$, which is what you're looking for.

You may also be misinterpreting the above equation. For $$p=2$$ and $$k=0$$, it says that:

$$\gamma(0) - \phi_1 \gamma(-1) - \phi_2 \gamma(-2) = \sigma^2$$

It does not say that $$\gamma(0) = \sigma^2$$. The terms where $$\gamma(i)$$ has a negative argument are not implicitly dropped or anything like that. Since $$\gamma$$ is symmetric, it also says that:

$$\gamma(0) - \phi_1 \gamma(1) - \phi_2 \gamma(2) = \sigma^2$$

You can combine this with the same equation for $$k=1$$ and $$k=2$$ to build a linear system with 3 equations and 3 unknowns, something like this:

$$\begin{pmatrix} \gamma(1) && \gamma(2) && 1 \\ \gamma(0) && \gamma(1) && 0 \\ \gamma(1) && \gamma(0) && 0 \end{pmatrix} \begin{pmatrix} \phi_1 \\ \phi_2 \\ \sigma^2 \end{pmatrix} = \begin{pmatrix} \gamma(0) \\ \gamma(1) \\ \gamma(2) \end{pmatrix}$$

More typically this is done in two steps, solving for the $$\phi_i$$ first in the smaller system:

$$\begin{pmatrix} \gamma(0) && \gamma(1) \\ \gamma(1) && \gamma(0) \\ \end{pmatrix} \begin{pmatrix} \phi_1 \\ \phi_2 \end{pmatrix} = \begin{pmatrix} \gamma(1) \\ \gamma(2) \end{pmatrix}$$

And then plugging in $$\phi_i$$ into the remaining equation to get $$\sigma^2$$.

Either way, the result is:

$$\sigma^2 = \left(\gamma(0) - \gamma(2)\right) \left( 1 + \frac{\gamma^2(1)-\gamma(0)\gamma(2)}{\gamma^2(1)-\gamma^2(0)}\right)$$

Substituting for the $$\hat{\gamma}(i)$$, you'll get to $$\hat{\sigma}^2 \approx 121$$.

• +1: This is a fantastic answer. It clarified a lot of things for me. I'd give it some extra points if I could. Commented Oct 8, 2021 at 11:32