# Times series with autocorrelated errors

I'm following the "Time Series Analysis and Its Applications With R Examples" from Shumway and Stoffer. In chapter 3.8 they talk about Regression with Autocorrelated Errors.

"They use the model of the form: $$y=Z*\beta+x$$

Where $$y=(y_1,...,y_n)'$$ and $$x=(x_1,...,x_n)'$$ are $$n \times 1$$ vectors, $$\beta=(\beta_1,...,\beta_n)'$$ is $$r \times 1$$, and $$Z=[z_1|z_2|...|z_n]'$$ is the $$n \times r$$ matrix composed of the input variables. Let $$\Gamma={\gamma_x(s,t)}$$, where $$\Gamma$$ is the variance-covariance matrix made from an ARMA(p,q) model, then $$\Gamma^{-\frac{1}{2}}y=\Gamma^{-\frac{1}{2}} Z \beta+ \Gamma^{-\frac{1}{2}}x$$, so that we can write the model as

$$y^\star=Z^\star \beta+ \delta,$$

where $$y^\star=\Gamma^{-\frac{1}{2}}y,Z^\star=\Gamma^{-\frac{1}{2}} Z$$, and $$\delta=\Gamma^{-\frac{1}{2}}x$$. Consequently, the covariance matrix of $$\delta$$ is the identity and the model is in the classical linear model form. It follows that the weighted estimate of $$\beta$$ is $$\hat{\beta_w}=(Z^{' \star} Z^\star)^{-1} Z^{' \star} y^\star =(Z' \Gamma^{-1} Z)^{-1} Z' \Gamma^{-1}y$$, and the variance-covariance matrix of the estimator is $$var(\hat{\beta_w}) = (Z' \Gamma^{-1} Z)^{-1}$$. If $$x_t$$ is white noise, then $$\Gamma = \sigma^2 I$$ and these results reduce to the usual least squares results."

During this process, I try to take the covariance of $$\delta$$ then I should get the following, $$cov(\delta)=cov(\Gamma^{-\frac{1}{2}}x) =\sigma^2 \Gamma^{-\frac{1}{2}} \Gamma \Gamma^{-\frac{1}{2}}=\sigma^2 I.$$ This is derived from the fact that $$cov(x)=\sigma^2 \Gamma$$ and using the property that $$cov(Ax)=A cov(x) A'$$ along with the property that $$(\Gamma^{-\frac{1}{2}})'=\Gamma^{-\frac{1}{2}}$$ since $$\Gamma^{-\frac{1}{2}}$$ is a symmetric matrix due to properties of covariance matrices. Additionally since $$\Gamma^{-1}$$ is formed from an ARMA(p,q) covariance matrix, the matrix has a $$\sigma^2$$ multiplier in it, so it is possible to factor it out from $$\Gamma$$ which provides the final result of $$cov(\delta)=\sigma^2 I$$.

So I have three questions that I am trying to understand,

1. Is my thought process correct in understanding this?
2. Does this mean that $$\hat{\beta_w}$$ doesn't depend on $$\sigma^2$$
3. Would this mean that the new transformed errors, $$\delta$$, are now iid such that $$\delta \sim N(0,\sigma^2)$$?

Thank you kindly

1. You have a minor error in your expression for $$cov(\delta)$$; the exponent on $$\Gamma$$ shouldn't be $$-1$$.

2. Yes, in OLS the coefficient estimates don't depend on $$\sigma^2$$. The variance-covariance matrix of the estimates does, but the estimates themselves do not.

3. There is no assumption of Normality required, just that the errors have a finite variance. If we added the assumption that $$x$$ was Normally distributed, with covariance matrix $$\Gamma$$, then, yes, $$\delta = \Gamma^{-1/2}x \sim \mathrm{N}(0, \sigma^2\mathrm{I})$$.

Note that it is unnecessary to make any distributional assumptions about $$x$$ (other than finite variance and independence from $$Z$$) for the asymptotic normality of $$\hat{\beta}$$ to hold; we do need some straightforward conditions on $$Z$$, e.g., $$Z$$ bounded, $$\lim_{T \to \infty} Z'Z/T$$ is finite and nonsingular.
The distribution of $$\delta$$ will depend on the distribution of $$x$$. For example, if $$x \sim \mathrm{MVt}(\nu, \Gamma)$$ (a multivariate $$t$$ distribution) then $$\delta$$ will have an $$\mathrm{MVt}(\nu, I)$$ distribution, i.e., each element of $$\delta \sim t(\nu)$$. Other cases may not work out so nicely, however.
• Thank you for the quick reply, I went ahead and fixed the error in 1. For question 2, yes I can see that when we have to take the variance of $\hat{\beta_w}$. Question 3, is still a bit confusing for me. If we don't assume that $x$ was normally distributed then what kind of distribution would $\delta$ follow? Thanks Jan 18 at 18:23