2
$\begingroup$

I have tried compute the autocovariance of the following process:

$$ Y_t = \beta_0+\beta_1t+\epsilon_t ~~~~~~~~~~~~,~ \epsilon_t \sim WN(0,\sigma^2) $$

I tried this way:

$$COV(Y_t, Y_{t-j}) = E[(Y_t - E(Y_t))(Y_{t-j}- E(Y_{t-j}))]$$ $$ = E[(\beta_0+\beta_1t+\epsilon_t-(\beta_0+\beta_1t))(\beta_0+\beta_1(t-j)+\epsilon_t-(\beta_0+\beta_1(t-j)))] $$

$$= E[\epsilon_t \cdot \epsilon_{t-j}] = 0$$

But this is not correct since if I simulate such a process with R and then I compute its acf, it is not zero. Where is my mistake?

$\endgroup$
1
  • $\begingroup$ Your process is not stationary, and you're not estimating the actual autocorrelation function when call R's acf function. $\endgroup$ Jan 26, 2017 at 10:42

2 Answers 2

2
$\begingroup$

The underlying problem is that your series is not stationary!

After you simulate your series $y_t$. R is computing the sample mean over time as:

$$ \bar{y} = \frac{1}{T} \sum_{\tau = 1}^T y_\tau $$

While with math, you computed the mean of $y_t$ as:

$$ E[y_t] = \beta_0 + \beta_1 t $$

Observe how completely different these two entities are! Because the process $\{y_t\}$ is not stationary, your sample mean taken over time is NOT an estimate of the population mean!

  • The population mean of $y_t$, that is $E[y_t] = \beta_0 + \beta_1 t$, is a function of time $t$.
  • On the other hand, the sample mean $\bar{y}$ is not an estimate of anything useful. It's a scalar, it doesn't depend on time $t$, and it goes to infinity as sample size $T$ goes to infinity.

So when R computes sample covariances etc... using $\bar{y}$, everything is already all fouled up! Look at how entirely non-sensical the calculation of sample covariance is:

$$\hat{\gamma}(k) = \frac{1}{T-1} \sum_{\tau } \left( y_\tau - \bar{y} \right) \left( y_{\tau - k} - \bar{y} \right) $$

This makes sense for a stationary process, but it's useless in this case. Taking averages over time of a non-stationary process is generally a huge error.

If you want to simulate to match your math answers, what can you do?

If you want to verify results such as $E[y_t] = \beta_0 + \beta_t$ by simulation, you want to conduct multiple draws from sample space $\Omega$ rather than advancing time $t$. You want to take averages over space $\Omega$ rather than time $t$.

If your process is non-stationary, it's still the case that for any time $t$, $y_t$ is a random variable and you can compute the sample mean etc... of that like any other random variable by taking IID draws from the sample space.

Sample averages over time only converge towards population averages across space in the case of stationary, ergodic processes.

$\endgroup$
1
$\begingroup$

The autocovariance function of this time series is indeed $0$ for $j \neq 0$ as you have computed quite correctly. So, the question is why R is giving a nonzero value for the acf. Unfortunately, acf is also an initialism for autocorrelation function which is defined as the value of $E[Y_tY_{t-j}]$, not $\operatorname{cov}(Y_t,Y_{t-j})$, in the engineering literature, and possibly in some parts of the statistical literature, and perhaps even in R. So, check what R believes acf means. If R does define the acf as $\operatorname{cov}(Y_t.Y_{t-j})$, but you are doing a simulation of the time series, then it is almost surely the case that the time-average value that your simulation gives does not exactly equal the ensemble average value of $0$. As long as the time average is small, nothing has gone wrong.

$\endgroup$
4
  • $\begingroup$ $E[Y_t, Y_{t-j}]$ is the same thing as $COV(Y_t, Y_{t-j})$ when $E[Y_t]=0$ as in this case. $\endgroup$
    – Archimede
    Jan 20, 2017 at 15:10
  • 1
    $\begingroup$ @Archimede What you say is true except for the assertion that "$E[Y_t] = 0$ as in this case." In this case, $E[Y_t]=E[\beta_0 + \beta_1t +\epsilon_t] = \beta_0 + \beta_1t$ which does not equal $0$ for all $t$ except when both $\beta_0$ and $\beta_1$ equal $0$. $\endgroup$ Jan 20, 2017 at 15:17
  • $\begingroup$ His process is not stationary... $\endgroup$ Jan 26, 2017 at 10:52
  • $\begingroup$ @MatthewGunn True, but irrelevant as far as the autocovariance function is concerned. cov$(Y_t, Y_{t-j})$ depends only on $j$ and not on $t$. It is the autocorrelation function that depends on both $j$ and $t$. $\endgroup$ Jan 28, 2017 at 21:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.