I'm trying to find a reference (including the full formula) for the following. If $X_n = a_1 X_{n-1} + \cdots a_p X_{n-p} + e(n)$ where $\{e(n)\}$ is a white noise, then

$$ X_n=g(e_0,e_1,\ldots,e_n)+\sum_{k=1}^{p}r_k^n\cdot Q_k,\ \mathrm{with}\ \begin{pmatrix} Q_1 \\ Q_2 \\ \vdots \\ Q_p \end{pmatrix} = V^{-1} \begin{pmatrix} X_0 \\ X_1 \\ \vdots \\ X_{p-1} \end{pmatrix} $$

where $V$ is the same matrix as in section 2 in this article, and $g$ is a linear function of $e_0,\ldots, e_n$ with some combination of the coefficients $a_1, \ldots, a_p$. For instance, if $p = 1$, we have $g(e_0,\ldots,e_n) = \sum_{k=0}^{n-1}a_1^k\cdot e_{n-k}$.

For an arbitrary $p$, the $k$-th term in the above sum (for the function $g$) is a polynomial of degree $k$, with $p$ variables $a_1, \ldots, a_p$. This formula also allows you to identify the auto-correlation structure, whether or not the time series is stationary or not.


The matrix $V$ is a $p \times p$ Vandermonde matrix (its elements are the powers of the roots $r_1,\ldots,r_p$ of the characteristic polynomial $x^p = a_1 x^{p-1} + \cdots + a_{p-1} x + a_p$. The formula assumes that these roots are different (no multiple root) otherwise the formula must be adjusted.

  • $\begingroup$ Does this have to do with AR vs. MA representations of a general ARMA model? There are known conditions when you can represent a general ARMA by a pure AR or a pure MA. Keywords: invertibility (for MA), causality (for AR). $\endgroup$ May 29, 2019 at 17:24
  • $\begingroup$ Not really, the formula in question is valid in the context of AR($p$), though of course, it applies to any more sophisticated process equivalent to an AR($p$). A more general version of this formula is probably available for ARMA. $\endgroup$ May 29, 2019 at 17:58
  • $\begingroup$ Does the formula have some name or interpretation for those of us who are not that math-headed? I wonder what its function is supposed to be and for what it can be useful. $\endgroup$ May 29, 2019 at 18:19
  • $\begingroup$ I added a "note" at the bottom of my question, not sure if it helps. The formula can be used to compute the lag-$k$ auto-correlations $\mbox{Correl}[X_n, X_{n-k}]$ ($k=1, 2, \cdots$) although there are other ways to solve this problem. $\endgroup$ May 29, 2019 at 18:23
  • $\begingroup$ The general idea is to come up with a unified framework that solves most statistical problems involving linear algebra: linear regression, PCA, pseudo-inverse matrices, Markov chains and so on, all using the same matrix $V$ in one way or another. See datasciencecentral.com/profiles/blogs/… $\endgroup$ May 29, 2019 at 18:27

1 Answer 1


I could not find a reference, but I worked a bit more on it, and here is the full formula (the details about the function $g$, for $n\geq $p):

$g(e_p, e_{p+1},\cdots,e_n) = \sum_{k=0}^{n-p}A_k e_{n-k}, \mbox{ with } A_k =\sum_{t=1}^p a_tA_{k-t}$.

It is now published in my recent article, here (see Appendix.) The function $g$ does not depend on $e_0, e_1, \cdots, e_{p-1}$, only on $e_p, e_{p+1},\cdots e_n$. The initial conditions for $A_k$ are

$A_0=1$ and $ A_{-1}=A_{-2}= \cdots =A_{-(p-1)} = 0$.

Illustration ($p=1$) including variance computation is provided in my article.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.