# How to estimate the autocorrelation function?

According to the Wikipedia article on Autocorrelation, the autocorrelation function can be estimated by:

$\hat{R}(k)=\frac{1}{(n-k) \sigma^2} \sum_{t=1}^{n-k} (X_t-\mu)(X_{t+k}-\mu)$

But if $\mu$ and $\sigma ^{2}$ are replaced by the standard formula for sample mean and sample variance, then this is a biased estimate.

So my question is:

1) If we estimate $\mu$ and $\sigma ^{2}$ by standard formulas, then $\hat{R}(k)$ will become biased. But is it common to use this? If yes, what is the rationale to use it even it is biased?

2) What is the better method?

3) When I'm studying time sereis, I notice my lectures put less emphasis on the property of bias. Is bias important in time serioes?

• Could you give a reference (except for the lecture notes, unless they are publicly available) for the claim that the estimator is biased? – Richard Hardy May 10 '17 at 13:30
• @RichardHardy Please see the Wikipedia article, it quotes from there. – Ken T May 10 '17 at 14:25

Background information on estimation bias: Before answering your specific questions in this post, it is worth exploring the bias that occurs in the sample auto-covariance estimator in this kind of problem. To do this, we will consider the corresponding summation-statistic that replaces the (unknown) true mean in the process with the sample mean:

$$\hat{S}(k) \equiv \sum_{t=1}^{n-k} (X_t - \bar{X})(X_{t+k} - \bar{X}) \quad \quad \quad \bar{X} \equiv \frac{1}{n} \sum_{t=1}^n X_t.$$

From the true autocorrelation function $\gamma$, we facilitate our analysis by defining the quantities:

$$\Pi_{n,k} \equiv \frac{1}{n(n-2k)} \sum_{i=1}^n \sum_{j=k+1}^{n-k} \gamma|i-j|.$$

This quantity is the average correlation value in the $n \times n$ correlation matrix, with one dimension trimmed on each side by $k$ entries. It arises in our later analysis because it is related to the sample mean and trimmed sample mean, which both arise in our analysis.

Given a stationary process with fixed mean and variance we have $\mathbb{Cov}(X_i, X_j) = \sigma^2 \gamma |i-j|$ so that $\mathbb{E}(X_i X_j) = \sigma^2 \gamma |i-j| + \mu^2$. Hence, the expected value of our summation statistic is:

\begin{aligned} \mathbb{E}(\hat{S}(k)) &= \mathbb{E} \Bigg( \sum_{t=1}^{n-k} (X_t - \bar{X})(X_{t+k} - \bar{X}) \Bigg) \\[8pt] &= \mathbb{E} \Bigg( \sum_{t=1}^{n-k} X_t X_{t+k} - \bar{X} \Bigg( \sum_{t=1}^{n-k} X_t + \sum_{t=1}^{n-k} X_{t+k} \Bigg) + (n-k) \bar{X}^2 \Bigg) \\[8pt] &= \mathbb{E} \Bigg( \sum_{t=1}^{n-k} X_t X_{t+k} - \bar{X} \Bigg( n\bar{X} + \sum_{t=k+1}^{n-k} X_t \Bigg) + (n-k) \bar{X}^2 \Bigg) \\[8pt] &= \mathbb{E} \Bigg( \sum_{t=1}^{n-k} X_t X_{t+k} - \bar{X} \sum_{t=k+1}^{n-k} X_t - k \bar{X}^2 \Bigg) \\[8pt] &= \mathbb{E} \Bigg( \sum_{t=1}^{n-k} X_t X_{t+k} \Bigg) - \mathbb{E} \Bigg( \bar{X} \sum_{t=k+1}^{n-k} X_t \Bigg) - k \mathbb{E} \Big( \bar{X}^2 \Big) \\[8pt] &= (n-k) \Big( \sigma^2 \gamma(k) + \mu^2 \Big) - (n-2k) \Big( \sigma^2 \Pi_{n,k} + \mu^2 \Big) - k \Big( \sigma^2 \Pi_{n,0} + \mu^2 \Big) \\[8pt] &= (n-k) \sigma^2 \gamma(k) - \sigma^2 \Big( (n-k) \Pi_{n,k} + k (\Pi_{n,0} - \Pi_{n,k}) \Big) \\[8pt] &= (n-k) \sigma^2 \Bigg[ \gamma(k) - \Bigg( \Pi_{n,k} + \frac{k}{n-k} (\Pi_{n,0} - \Pi_{n,k}) \Bigg) \Bigg]. \\[8pt] \end{aligned}

Now, consider the auto-covariance estimator:

$$\hat{C}(k) \equiv \frac{1}{n-k} \sum_{t=1}^{n-k} (X_t - \bar{X})(X_{t+k} - \bar{X}).$$

From the above results we have:

$$\frac{\mathbb{E} (\hat{C}(k))}{\sigma^2} = \gamma(k) - \Bigg( \Pi_{n,k} + \frac{k}{n-k} (\Pi_{n,0} - \Pi_{n,k}) \Bigg).$$

This expression shows that there is a clear bias term in our analysis. However, for most stationary processes of interest, the auto-correlation dissipates as observations become farther apart in time. This means that when $n$ is large, the average auto-correlation terms $\Pi_{n,k}$ become small, and so the bias term (the second term in the expression) also becomes small.

1) It is common for analysts to use these standard estimators by substituting in the sample mean and variance. So long as the auto-correlation structure meets standard requirements (i.e., tends to dissipate as observations get farther apart in time), these are asymptotically unbiased and also consistent. This means that when $n$ is large they will be reasonable estimators. Although the estimator is biased, the bias is small for large $n$.