6
$\begingroup$

Brockwell/Davis, Introduction to Time Series and Forecasting, p. 40, write (notation slightly adapted; please refer to screenshot below)

The best linear predictor $l(Y_{T})=aY_{T}+b$ for a stationary time series $Y_{T+h}$ based on $Y_{T}$ minimizes $E[Y_{T+h}-aY_T-b]^2$ and is given by, for $\mu$, $\gamma_h$ and $\rho_h$ the mean and $h$-order autocovariance/autocorrelation, $$ l(Y_T)=\mu+\rho_h(Y_{T}-\mu), $$ so that the optimal $b=\mu(1-\rho_h)$ and $a=\rho_h$. The MSE is $$ E[Y_{T+h}-l(Y_{T})]^2=\gamma_0(1-\rho_h^2). $$ I can follow these steps up until here.

BD then go on to write:

This calculation shows that [...] prediction of $Y_{T+h}$ in terms of $Y_T$ is more accurate as $|\rho_h|$ becomes closer to 1, and in the limit as $\rho_h → \pm1$ [...] the corresponding mean squared error approaches 0.

I am not so sure what to make of this quote: consider for example a zero-mean AR(1) process with coefficient $\rho$, $Y_t=\rho Y_{t-1}+\epsilon_t$, and error variance $\sigma^2$, so that $\gamma_h=\sigma^2\rho^h/(1-\rho^2)$ and $$\rho_h=\frac{\gamma_h}{\gamma_0}=\frac{\sigma^2\rho^h/(1-\rho^2)}{\sigma^2/(1-\rho^2)}=\rho^h.$$

Then $$ MSE_{h,\rho}=\sigma^2\frac{1-\rho^{2h}}{1-\rho^2} $$ In this AR(1) case, $\rho_h\to1$ iff $\rho\to1$. By l'Hopital's rule this tends to $$ \lim_{\rho\to1}MSE_{h,\rho}=h\sigma^2, $$ which is also the MSE of an $h$-step ahead forecast of a random walk $Y_t=Y_{t-1}+\epsilon_t$, $$E(Y_{T+h}-Y_T)^2=E\left(\sum_{j=T+1}^{T+h}\epsilon_j\right)^2=h\sigma^2.$$

This is intuitive to me as the AR(1) approaches a random walk as $\rho\to1$, and the AR(1) $h$-step ahead forecasts $\rho^hY_T$ also approach those of a RW, $Y_T$.

In particular, as the variance $\gamma_0$ of the process is also affected by changes in the persistence $\rho$ of the process, this result suggests that the MSE does not converge to zero and also that it would not be the case that persistent processes are more predictable: An evaluation of

h <- 1:20
rho <- seq(.9, .99, by=.01)
MSE.AR <- (1-outer(rho, 2*h, "^"))/(1-rho^2)
MSE.RW <- h

matplot(h, t(MSE.AR), type="l", lwd=1, col=seq_len(length(rho)), lty=seq_len(length(rho)), ylim=c(0,max(h)))
lines(h, MSE.RW, lwd=5, lty=1, col="black")
legend("topleft", legend=c(rho, "RW"), col=c(seq_len(length(rho)), "black"), lty=c(seq_len(length(rho)),1), lwd=c(rep(1, length(rho)),5))

reveals that the MSE according to this result increases in the persistence of the process:

enter image description here

I am reluctant to claim, though, that BD make a mistake. Is anything wrong with my example?

Original discussion in BD (note in particular that their $\sigma^2$ is my $\gamma_0$): enter image description here

enter image description here

Note: This was to be part of an answer here, but actually gave rise to a question of mine, as posted here.

UPDATE:

In personal communication with Richard Davis, he states (my summary) that for a fixed $h$ and a stationary process (so no AR(1) approaching a random walk), if $\rho_h \to 1$, then you can perfectly predict $X_h$ from $X_0$. This is related to the relationship between any two random variables $X$ & $Y$. If their correlation is 1, then they are linearly related and one can be perfectly predicted from the other.

Hence, in the context of my example, it seems we would need $\sigma^2\to0$ to get such perfect correlation, in which case both $\gamma_0$ and $(1-\rho_h^2)$ in $\gamma_0(1-\rho_h^2)$ would go to zero.

$\endgroup$
14
  • $\begingroup$ There are predictability and forecastability tags (which I have proposed to merge), perhaps you want to add one. $\endgroup$ Commented May 5 at 9:55
  • $\begingroup$ Thanks, I have done so! $\endgroup$ Commented May 5 at 10:09
  • $\begingroup$ I think the crucial statement to investigate is: $\rho_h\to 1$ iff $\rho\to 1$. $\endgroup$ Commented May 5 at 11:12
  • 1
    $\begingroup$ Christoph: That's interesting and my Brockwell and Davis book is in storage. What are $a$ and $b$ in B&D's example ? Is $b$ equal to $(1-\rho_h) \mu$ and $a = \rho_h$. Thanks and sorry for confusion. $\endgroup$
    – mlofton
    Commented May 6 at 10:15
  • 1
    $\begingroup$ Thank you for getting to the bottom of this! $\endgroup$ Commented May 13 at 8:39

1 Answer 1

2
$\begingroup$

As things stand, I cannot see how the prediction MSE for any process that regularly has an error term $\epsilon_t$ with variance $\sigma^2$ coming in, independently of what happened before, can ever be smaller than $\sigma^2$.

So as far as I can see you are right and what BD claim there is wrong (at least some kind of typo). Obviously there may be subtleties in definitions hidden elsewhere in the book, but from the presented material and what I have seen and know about AR(1), this is how it is.

$\endgroup$
2
  • $\begingroup$ Hm. That's intriguing. I have one thing in mind. Brockwell passed away last year. But Professor Davis is still active. So, we might hope an email on this could shed enough light on this. $\endgroup$ Commented May 7 at 10:15
  • $\begingroup$ Indeed, the MSE is $\sigma^2$ also according to the expressions I found for the AR(1) at $h=1$, and increases from there on. Thanks for the idea about the email, I will consider that after having waited here for a little more! $\endgroup$ Commented May 7 at 11:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.