A question about serial correlation

Question

I'm new to Time Series and I'm confused by some assumptions relating the classical regression model and time series. Econometrics, Fumio Hayashi's book, when introducing the classical regression model, makes some caveats for the case of time series. For example, in page 45:

But, as emphasized in Section 1.1, the strict exogeneity assumption is not satisfied in time-series models typically encountered in econometrics, and serial correlation is an issue that arises only in time-series models.

Moreover, the linear model is given by $$y_i = x_i'\beta + \varepsilon_i, \quad i = 1,...,n$$

and the Assumption 1.4 (spherical error variance) is: $$E[\varepsilon_i^2|X]=\sigma^2 > 0, \quad E[\varepsilon_i \varepsilon_j |X] =0 $$ where $X = [\mathbf{x}_1,...,\mathbf{x}_K]_{n\times K}$ matrix. As said earlier, the most time series models do not satisfy this hypothesis, because of the serial correlation ($E(\varepsilon_i \varepsilon_j)\neq 0$). But for me, I can't see that most time series have serial correlation. For example, AR(1) processes are: $$y_t = \rho y_{t-1} + \varepsilon_t$$ with $(\varepsilon_t)$ being i.i.d.. So, I think that in this case, I don't have serial correlation, because the error is i.i.d. And in many other time series models (MA, ARMA) it seems to me that we have i.i.d errors.

So in what cases (examples) do we have serial correlation?

Answer 1

In the models that you name the error term is i.i.d. by assumption. Indeed, all time series models that I can now remember assume the error (appropriately defined*) to be i.i.d. or at least uncorrelated. However, the model need not approximate the data generating process (DGP) perfectly. E.g. if the DGP is ARMA(10,10) but we approximate it with a more parsimonious ARMA(2,2), the resulting error will be autocorrelated (serially correlated) and not i.i.d. Even so the reduction in estimation variance by replacing ARMA(10,10) by ARMA(2,2) may be so great that it will more than offset model bias, so that our ARMA(2,2) would do better in prediction than an ARMA(10,10) with estimated (rather than actual) coefficients. (The actual coefficients are of course not accessible to us.)

Another case could be regression with ARMA errors such as \begin{aligned} y_t &= \beta_0+\beta_1 x_t+u_t \\ u_t &= \varphi_1 u_{t-1} + \varepsilon_t + \theta_1\varepsilon_{t-1} \end{aligned} with $\varepsilon_t\sim\text{i.i.d.}$ Here you can explicitly see that $u_t$ is autocorrelated, and so using just $y_t = \beta_0+\beta_1 x_t+u_t$ and assuming $u_t\sim\text{i.i.d.}$ would violate the i.i.d. assumption. (It is easy to notice the problem with the i.i.d. assumption on $u_t$ when the model is explicitly stated with an ARMA equation for $u_t$, but it took some time to figure this out in the 20. century when people were working with simpler models without explicit equations for the error term.) For a more in-depth presentation of the model and an example (US personal consumption vs. income), see Hyndman "The ARIMAX model muddle" and section 9.2 from Hyndman & Athanasopoulos "Forecasting: Principles and Practice".

*E.g. a GARCH model assumes the standardized error to be i.i.d. while the raw error is conditionally heteroskedastic, though not autocorrelated.

Answer 2

Hi: you do have serial correlation in the AR(1) but it's kind of masked by the way it's written.

$y_t = \phi y_{t-1} + \epsilon_t$ is one way of writing it. Using the lag operator, it can also be written as:

$y_t = \sum_{i=0}^{\infty} \phi^{i} \epsilon_{t-i}$.

Writing it this way, shows that the response, $y_t$, is dependent on all previous shocks in a geometrically declining manner. So, although the $\epsilon_{i}$ are independent, the $y_{t}$ are serially correlated so the model does not satisfy the usual OLS assumptions.