I'm following the "Time Series Analysis and Its Applications With R Examples" from Shumway and Stoffer. In chapter 3.8 they talk about Regression with Autocorrelated Errors.

"They use the model of the form: $$y=Z*\beta+x$$

Where $y=(y_1,...,y_n)'$ and $x=(x_1,...,x_n)'$ are $n \times 1$ vectors, $\beta=(\beta_1,...,\beta_n)'$ is $r \times 1$, and $Z=[z_1|z_2|...|z_n]'$ is the $n \times r$ matrix composed of the input variables. Let $\Gamma={\gamma_x(s,t)}$, where $\Gamma$ is the variance-covariance matrix made from an ARMA(p,q) model, then $\Gamma^{-\frac{1}{2}}y=\Gamma^{-\frac{1}{2}} Z \beta+ \Gamma^{-\frac{1}{2}}x$, so that we can write the model as

$$y^\star=Z^\star \beta+ \delta,$$

where $y^\star=\Gamma^{-\frac{1}{2}}y,Z^\star=\Gamma^{-\frac{1}{2}} Z$, and $\delta=\Gamma^{-\frac{1}{2}}x$. Consequently, the covariance matrix of $\delta$ is the identity and the model is in the classical linear model form. It follows that the weighted estimate of $\beta$ is $\hat{\beta_w}=(Z^{' \star} Z^\star)^{-1} Z^{' \star} y^\star =(Z' \Gamma^{-1} Z)^{-1} Z' \Gamma^{-1}y$, and the variance-covariance matrix of the estimator is $var(\hat{\beta_w}) = (Z' \Gamma^{-1} Z)^{-1}$. If $x_t$ is white noise, then $\Gamma = \sigma^2 I$ and these results reduce to the usual least squares results."

During this process, I try to take the covariance of $\delta$ then I should get the following, $$cov(\delta)=cov(\Gamma^{-\frac{1}{2}}x) =\sigma^2 \Gamma^{-\frac{1}{2}} \Gamma \Gamma^{-\frac{1}{2}}=\sigma^2 I.$$ This is derived from the fact that $cov(x)=\sigma^2 \Gamma$ and using the property that $cov(Ax)=A cov(x) A'$ along with the property that $(\Gamma^{-\frac{1}{2}})'=\Gamma^{-\frac{1}{2}}$ since $\Gamma^{-\frac{1}{2}}$ is a symmetric matrix due to properties of covariance matrices. Additionally since $\Gamma^{-1}$ is formed from an ARMA(p,q) covariance matrix, the matrix has a $\sigma^2$ multiplier in it, so it is possible to factor it out from $\Gamma$ which provides the final result of $cov(\delta)=\sigma^2 I$.

So I have three questions that I am trying to understand,

  1. Is my thought process correct in understanding this?
  2. Does this mean that $\hat{\beta_w}$ doesn't depend on $\sigma^2$
  3. Would this mean that the new transformed errors, $\delta$, are now iid such that $\delta \sim N(0,\sigma^2)$?

Thank you kindly


1 Answer 1

  1. You have a minor error in your expression for $cov(\delta)$; the exponent on $\Gamma$ shouldn't be $-1$.

  2. Yes, in OLS the coefficient estimates don't depend on $\sigma^2$. The variance-covariance matrix of the estimates does, but the estimates themselves do not.

  3. There is no assumption of Normality required, just that the errors have a finite variance. If we added the assumption that $x$ was Normally distributed, with covariance matrix $\Gamma$, then, yes, $\delta = \Gamma^{-1/2}x \sim \mathrm{N}(0, \sigma^2\mathrm{I})$.

Updated in response to comments:

Note that it is unnecessary to make any distributional assumptions about $x$ (other than finite variance and independence from $Z$) for the asymptotic normality of $\hat{\beta}$ to hold; we do need some straightforward conditions on $Z$, e.g., $Z$ bounded, $\lim_{T \to \infty} Z'Z/T$ is finite and nonsingular.

The distribution of $\delta$ will depend on the distribution of $x$. For example, if $x \sim \mathrm{MVt}(\nu, \Gamma)$ (a multivariate $t$ distribution) then $\delta$ will have an $\mathrm{MVt}(\nu, I)$ distribution, i.e., each element of $\delta \sim t(\nu)$. Other cases may not work out so nicely, however.

  • $\begingroup$ Thank you for the quick reply, I went ahead and fixed the error in 1. For question 2, yes I can see that when we have to take the variance of $\hat{\beta_w}$. Question 3, is still a bit confusing for me. If we don't assume that $x$ was normally distributed then what kind of distribution would $\delta$ follow? Thanks $\endgroup$ Jan 18, 2023 at 18:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.