I've been reading into how to minimize objective functions and I am curious about the following, I have a model $y=X \beta +\epsilon$ where $E[\epsilon|X]=0$ and $Var[\epsilon|X]=\Sigma$ where $\Sigma$ is not diagonal and is a covariance matrix. $\beta$ can be estimated by least squares to get, $$\hat{\beta}=(X^T \Sigma^{-1}X)^{-1}X^T\Sigma^{-1}y$$.

So this comes from the minimization of $(y-X\beta)^T \Sigma^{-1}(y-X\beta)$. Now if I assume that $\epsilon \sim AR(2)$ then $Var[\epsilon|X]=\Sigma=Cov(AR(2))$ making it a covariance matrix with respect to AR(2). So my question is, is there any way to decompose $\Sigma$ in the objective function such that it can be written in the form of the AR(2) coefficients and its variance so that the objective function can be minimized with respect to variance, and AR(2) coefficients if $y, \beta$, and $X$ are known.

Thank you

  • 1
    $\begingroup$ Are you only interested in a "decomposition" of $\Sigma$, if so, what exactly do you mean by "decomposition"? Or do you want to understand the complete procedure of this optimization problem? Or do you just want to know an implementation you can use? $\endgroup$
    – frank
    Sep 30, 2022 at 5:21
  • $\begingroup$ Hi Frank, I actually understand most of the procedures of this optimization, however, I am curious in knowing if anything can be done with $\Sigma$, either rewritten in a different way say through QR decomposition or if it is legitimate to rewrite it as $\sigma^2_w* \Sigma$ since originally $\Sigma$ is calculated using the auto-covariance function which relies on $\sigma^2_w, \phi_1, \phi_2$ where $\sigma^2_w$ is just the variance of white noise, and $\phi_1, \phi_2$ are the coefficients values of the AR(2) model. $\endgroup$ Sep 30, 2022 at 16:29

1 Answer 1


You can do linear regression for $y = X\beta + \epsilon$ using a covariance matrix for $\epsilon$ that belongs to an AR(2) model, or to any ARMA(p, q) model for that matter.

You can e.g. use the gls() function from the nlme R-package. It provides for various types of correlations, among them ARMA correlation (look for corARMA here). There, you provide the coefficients of the ARMA model which are then used by gls() to create the appropriate covariance matrix and execute the optimization.

  • $\begingroup$ THanks, Frank, I'll have to check the gls() out as I haven't had to use it yet. $\endgroup$ Oct 3, 2022 at 13:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.