0
$\begingroup$

Say I have a VAR(p) model without any noise (a multidimensional AR model without noise). How would I go about calculating the coefficients that are MSE optimal? Is there an extension to the Yule Walker equations for multidimensional data?

Edit: Maybe an equation like the following would help?

$\hat Y_{k+1}^{}=AY_{k} $

where any general $\ Y_{i}=[Y_{1} \ Y_{2}\ Y_{3}\ Y_{4}\ ...... \ Y_{p}]^{T} $ and I want to estimate the optimal A in MSE sense. Such a problem should fit under a Weiner filtering approach in my opinion. Would using the matrix orthogonality principle be sufficient to solve this?

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ Contrary to what you are implying, an AR model does contain a noise component. E.g. in an AR(1): $y_t=c+\varphi_1 y_{t-1}+\varepsilon_t$, it is $\varepsilon_t$. Given that, can you specify more precisely what your VAR(p) model without any noise is? $\endgroup$
    – Richard Hardy
    May 13, 2022 at 7:55
  • $\begingroup$ @RichardHardy I updated my question and made it clearer. $\endgroup$
    – Harduin
    May 14, 2022 at 3:34

1 Answer 1

Reset to default
0
$\begingroup$

$Y_{k+1}=AY_{k}$ where any general $\ Y_{i}=[Y_{1} \ Y_{2}\ Y_{3}\ Y_{4}\ ...... \ Y_{p}]^{T} $ and I want to estimate the optimal A in MSE sense.

So you have an exact, deterministic, linear relationship. The only thing that is missing is the coefficient values in $A$. Then equation-by-equation OLS will return you exact estimates, and they will be MSE-optimal.

I am trying to figure out a problem where this is perhaps not even $ Y_{k+1}$ but rather maybe just an estimate of it.

If you change your problem by making $Y_{k+1}$ only observable with some error while $Y_1,\dots,Y_k$ are observable without error, you get a multivariate multiple regression. There is no MSE-optimal estimator for that. If you were to restrict your attention to linear estimators and the error variance were given, it would be a form of ridge regression (a univariate case is discussed in Dave Giles' blog "A Regression "Estimator" that Minimizes MSE". But that is perhaps too restrictive.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Why is there no MSE optimal answer for that? Shouldnt the Orthogonality Principle result in a system of equations that can solve this? I have a picture of my approach but I am unsure whether a non latex solution would be appropriate on this forum. $\endgroup$
    – Harduin
    May 14, 2022 at 19:31
  • $\begingroup$ @Harduin, I am not familiar with the orthogonality principle from before, but from a casual read it seems it might ignore estimation imprecision and so would not be directly applicable in practice. Have you checked out the blog post? It deals with these issues. $\endgroup$
    – Richard Hardy
    May 14, 2022 at 19:36
  • $\begingroup$ And since you are using signal processing terminology, you may be interested in learning there is Signal Processing Stack Exchange. $\endgroup$
    – Richard Hardy
    May 14, 2022 at 19:50
  • $\begingroup$ Thank you for the help. I'll look into this. $\endgroup$
    – Harduin
    May 15, 2022 at 0:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.