# General formula for AR($p$) auto-regressive time series

I'm trying to find a reference (including the full formula) for the following. If $$X_n = a_1 X_{n-1} + \cdots a_p X_{n-p} + e(n)$$ where $$\{e(n)\}$$ is a white noise, then

$$X_n=g(e_0,e_1,\ldots,e_n)+\sum_{k=1}^{p}r_k^n\cdot Q_k,\ \mathrm{with}\ \begin{pmatrix} Q_1 \\ Q_2 \\ \vdots \\ Q_p \end{pmatrix} = V^{-1} \begin{pmatrix} X_0 \\ X_1 \\ \vdots \\ X_{p-1} \end{pmatrix}$$

where $$V$$ is the same matrix as in section 2 in this article, and $$g$$ is a linear function of $$e_0,\ldots, e_n$$ with some combination of the coefficients $$a_1, \ldots, a_p$$. For instance, if $$p = 1$$, we have $$g(e_0,\ldots,e_n) = \sum_{k=0}^{n-1}a_1^k\cdot e_{n-k}$$.

For an arbitrary $$p$$, the $$k$$-th term in the above sum (for the function $$g$$) is a polynomial of degree $$k$$, with $$p$$ variables $$a_1, \ldots, a_p$$. This formula also allows you to identify the auto-correlation structure, whether or not the time series is stationary or not.

Note

The matrix $$V$$ is a $$p \times p$$ Vandermonde matrix (its elements are the powers of the roots $$r_1,\ldots,r_p$$ of the characteristic polynomial $$x^p = a_1 x^{p-1} + \cdots + a_{p-1} x + a_p$$. The formula assumes that these roots are different (no multiple root) otherwise the formula must be adjusted.

• Does this have to do with AR vs. MA representations of a general ARMA model? There are known conditions when you can represent a general ARMA by a pure AR or a pure MA. Keywords: invertibility (for MA), causality (for AR). – Richard Hardy May 29 '19 at 17:24
• Not really, the formula in question is valid in the context of AR($p$), though of course, it applies to any more sophisticated process equivalent to an AR($p$). A more general version of this formula is probably available for ARMA. – Vincent Granville May 29 '19 at 17:58
• Does the formula have some name or interpretation for those of us who are not that math-headed? I wonder what its function is supposed to be and for what it can be useful. – Richard Hardy May 29 '19 at 18:19
• I added a "note" at the bottom of my question, not sure if it helps. The formula can be used to compute the lag-$k$ auto-correlations $\mbox{Correl}[X_n, X_{n-k}]$ ($k=1, 2, \cdots$) although there are other ways to solve this problem. – Vincent Granville May 29 '19 at 18:23
• The general idea is to come up with a unified framework that solves most statistical problems involving linear algebra: linear regression, PCA, pseudo-inverse matrices, Markov chains and so on, all using the same matrix $V$ in one way or another. See datasciencecentral.com/profiles/blogs/… – Vincent Granville May 29 '19 at 18:27

I could not find a reference, but I worked a bit more on it, and here is the full formula (the details about the function $$g$$, for $$n\geq$$p):
$$g(e_p, e_{p+1},\cdots,e_n) = \sum_{k=0}^{n-p}A_k e_{n-k}, \mbox{ with } A_k =\sum_{t=1}^p a_tA_{k-t}$$.
It is now published in my recent article, here (see Appendix.) The function $$g$$ does not depend on $$e_0, e_1, \cdots, e_{p-1}$$, only on $$e_p, e_{p+1},\cdots e_n$$. The initial conditions for $$A_k$$ are
$$A_0=1$$ and $$A_{-1}=A_{-2}= \cdots =A_{-(p-1)} = 0$$.
Illustration ($$p=1$$) including variance computation is provided in my article.