10
$\begingroup$

I'm reading a time series analysis book and the formula for sample autocovariance is defined in the book as:

$$\widehat{\gamma}(h) = n^{-1}\displaystyle\sum_{t=1}^{n-h}(x_{t+h}-\bar{x})(x_t-\bar{x})$$

with $\widehat{\gamma}(-h) = \widehat{\gamma}(h)\;$ for $\;h = 0,1, ..., n-1$. $\bar{x}$ is the mean.

Can someone explain intuitively why we divide the sum by $n$ and not by $n-h$? The book explains that this is because the formula above is a non-negative definite function and so dividing by $n$ is preferred, but this isn't clear to me. Can someone maybe prove this or show an example or something?

To me the intuitive thing at first would be to divide by $n-h$. Is this an unbiased or biased estimator of autocovariance?

$\endgroup$
  • 1
    $\begingroup$ If your time series is exactly $x_1, x_2, \ldots, x_n$ with all other $x_i$, $i < 1$ or $i >n$ being unknown, then the sum must necessarily stop at $t=n-h$ when $x_{t+h}=x_n$ occurs in the sum: the next term (for $t=n-h+1$) that would be included in the sum would have $x_{n-h+1+h}=x_{n+1}$ in it, and $x_{n+1}$ is not part of the sample. $\endgroup$ – Dilip Sarwate Apr 16 '13 at 11:58
  • $\begingroup$ @Dilip I don't think that's the issue: the question concerns whether to divide by $n$ or $n-h$ in the definition of $\hat{\gamma}$. $\endgroup$ – whuber Apr 16 '13 at 15:06
14
$\begingroup$

$\widehat{\gamma}$ is used to create covariance matrices: given "times" $t_1, t_2, \ldots, t_k$, it estimates that the covariance of the random vector $X_{t_1}, X_{t_2}, \ldots, X_{t_k}$ (obtained from the random field at those times) is the matrix $\left(\widehat{\gamma}(t_i - t_j), 1 \le i, j \le k\right)$. For many problems, such as prediction, it is crucial that all such matrices be nonsingular. As putative covariance matrices, obviously they cannot have any negative eigenvalues, whence they must all be positive-definite.

The simplest situation in which the distinction between the two formulas

$$\widehat{\gamma}(h) = n^{-1}\sum_{t=1}^{n-h}(x_{t+h}-\bar{x})(x_t-\bar{x})$$

and

$$\widehat{\gamma}_0(h) = (n-h)^{-1}\sum_{t=1}^{n-h}(x_{t+h}-\bar{x})(x_t-\bar{x})$$

appears is when $x$ has length $2$; say, $x = (0,1)$. For $t_1=t$ and $t_2 = t+1$ it's simple to compute

$$\widehat{\gamma}_0 = \left( \begin{array}{cc} \frac{1}{4} & -\frac{1}{4} \\ -\frac{1}{4} & \frac{1}{4} \end{array} \right),$$

which is singular, whereas

$$\widehat{\gamma} = \left( \begin{array}{cc} \frac{1}{4} & -\frac{1}{8} \\ -\frac{1}{8} & \frac{1}{4} \end{array} \right)$$

which has eigenvalues $3/8$ and $1/8$, whence it is positive-definite.

A similar phenomenon happens for $x = (0,1,0,1)$, where $\widehat{\gamma}$ is positive-definite but $\widehat{\gamma}_0$--when applied to the times $t_i = (1,2,3,4)$, say--degenerates into a matrix of rank $1$ (its entries alternate between $1/4$ and $-1/4$).

(There is a pattern here: problems arise for any $x$ of the form $(a,b,a,b,\ldots,a,b)$.)

In most applications the series of observations $x_t$ is so long that for most $h$ of interest--which are much less than $n$--the difference between $n^{-1}$ and $(n-h)^{-1}$ is of no consequence. So in practice the distinction is no big deal and theoretically the need for positive-definiteness strongly overrides any possible desire for unbiased estimates.

$\endgroup$
  • 1
    $\begingroup$ I think its important to note that both estimators are biased estimators, even if you divide it by n-h. $\endgroup$ – Ran Jun 10 '13 at 10:56
  • $\begingroup$ @Ran Although you are correct that these estimators are biased, I disagree that this is an important issue: as mentioned in the last paragraph, a small amount of bias is the least of anybody's worries. The unbiased estimator, using $(n-h-1)^{-1}$, scarcely differs from $\widehat{\gamma}$ or $\widehat{\gamma}_0$. $\endgroup$ – whuber Jun 10 '13 at 12:57
  • 1
    $\begingroup$ Very nice answer +1. Perhaps it is useful to add the point that $\mathbb{V} \hat{\gamma}_0(h) = O(1/(n-h))$, while $\mathbb{V} \hat{\gamma}(h) = O(1/n)$, so when $h$ is close to $n$, the estimator $\hat{\gamma}_0(h)$ can be erratic, while $\hat{\gamma}(h)$ will have uniformly small sampling fluctuations $\forall h$. See eg Priestly (1981) "Spectral Analysis and Time Series" p324 for a detailed discussion of this point $\endgroup$ – Colin T Bowers Sep 6 '13 at 0:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.