1
$\begingroup$

I've been reading into how to minimize objective functions and I am curious about the following, I have a model $y=X \beta +\epsilon$ where $E[\epsilon|X]=0$ and $Var[\epsilon|X]=\Sigma$ where $\Sigma$ is not diagonal and is a covariance matrix. $\beta$ can be estimated by least squares to get, $$\hat{\beta}=(X^T \Sigma^{-1}X)^{-1}X^T\Sigma^{-1}y$$.

So this comes from the minimization of $(y-X\beta)^T \Sigma^{-1}(y-X\beta)$. Now if I assume that $\epsilon \sim AR(2)$ then $Var[\epsilon|X]=\Sigma=Cov(AR(2))$ making it a covariance matrix with respect to AR(2). So my question is, is there any way to decompose $\Sigma$ in the objective function such that it can be written in the form of the AR(2) coefficients and its variance so that the objective function can be minimized with respect to variance, and AR(2) coefficients if $y, \beta$, and $X$ are known.

Thank you

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ Are you only interested in a "decomposition" of $\Sigma$, if so, what exactly do you mean by "decomposition"? Or do you want to understand the complete procedure of this optimization problem? Or do you just want to know an implementation you can use? $\endgroup$
    – frank
    Commented Sep 30, 2022 at 5:21
  • $\begingroup$ Hi Frank, I actually understand most of the procedures of this optimization, however, I am curious in knowing if anything can be done with $\Sigma$, either rewritten in a different way say through QR decomposition or if it is legitimate to rewrite it as $\sigma^2_w* \Sigma$ since originally $\Sigma$ is calculated using the auto-covariance function which relies on $\sigma^2_w, \phi_1, \phi_2$ where $\sigma^2_w$ is just the variance of white noise, and $\phi_1, \phi_2$ are the coefficients values of the AR(2) model. $\endgroup$
    – Warhawk1987
    Commented Sep 30, 2022 at 16:29

1 Answer 1

Reset to default
0
$\begingroup$

You can do linear regression for $y = X\beta + \epsilon$ using a covariance matrix for $\epsilon$ that belongs to an AR(2) model, or to any ARMA(p, q) model for that matter.

You can e.g. use the gls() function from the nlme R-package. It provides for various types of correlations, among them ARMA correlation (look for corARMA here). There, you provide the coefficients of the ARMA model which are then used by gls() to create the appropriate covariance matrix and execute the optimization.

Improve this answer
$\endgroup$
1
  • $\begingroup$ THanks, Frank, I'll have to check the gls() out as I haven't had to use it yet. $\endgroup$
    – Warhawk1987
    Commented Oct 3, 2022 at 13:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.