4
$\begingroup$

I know that an ARMA(p, q) refers to the model with p autoregressive terms and q moving-average terms. For example:

$$ X_t = c + \varepsilon_t + \sum_{i=1}^p \varphi_i X_{t-i} + \sum_{i=1}^q \theta_i \varepsilon_{t-i} $$

where $\varepsilon_{t} \sim WN(0,\sigma^2)$. That is, $E(\varepsilon_t) = 0$ and $E(\varepsilon_t \varepsilon_{t-j}) = 0$, for $j\neq 0$. Equivalently, $E(\varepsilon_t | \varepsilon_{t-j}) = 0$.

But in the books I've read, it doesn't tell me anything about the relationship between all the lags of $\varepsilon_t$ and all the lags of $X_t$. For example:

  1. Are $X_t$ and $\varepsilon_t$ independent? And how about $X_t$ and $\varepsilon_{t-j}$?
  2. If not, are they mean independent? That is: $E[X_t | \varepsilon_{t-j}] = X_t$?
  3. Are they not correlated? That is, $cov(X_t,\varepsilon_{t-j})=0$?

Some help?

Improve this question
$\endgroup$
7
  • 1
    $\begingroup$ A more instructive set of questions would inquire about the relationship between $X_{t-1}$ and $\varepsilon_t.$ $\endgroup$
    – whuber ♦
    Commented Dec 29, 2021 at 18:59
  • 1
    $\begingroup$ Yes: that follows from the definition of a white noise process. $\endgroup$
    – whuber ♦
    Commented Dec 29, 2021 at 21:19
  • 1
    $\begingroup$ @whuber, I would think a white noise process is not defined in reference to other processes su as $X$ (the ARMA process), but an ARMA process can be defined in a generative way in reference to a white noise process (telling how we generate $X$ from $\varepsilon$ where the latter is already defined). So I would think zero correlation of $X_{t-1}$ with $\varepsilon_t$ follows from the latter definition, not the former and that your comment is wrong. Where is my mistake? $\endgroup$
    – Richard Hardy
    Commented Dec 30, 2021 at 5:45
  • 1
    $\begingroup$ @Richard Imagine the $\varepsilon_t$ came from something other than a white noise process. The conclusion (that $X_{t-1}$ and $\varepsilon_t$ are uncorrelated) would no longer follow. Ergo, that conclusion must rest--at least in part--on properties of the white noise process. $\endgroup$
    – whuber ♦
    Commented Dec 30, 2021 at 15:27
  • 1
    $\begingroup$ @whuber, thank you, that makes sense. $\endgroup$
    – Richard Hardy
    Commented Dec 30, 2021 at 16:08

1 Answer 1

Reset to default
2
$\begingroup$
  1. No, $X_t$ and $\varepsilon_t$ are dependent. $X_t$ is a linear function of $\varepsilon_t$, and $X_t$ is positively correlated with $\varepsilon_t$: \begin{aligned} \text{Cov}(X_t,\varepsilon_t) &= \text{Cov}([c + \sum_{i=1}^p \varphi_i X_{t-i} + \sum_{i=1}^q \theta_i \varepsilon_{t-i}] + \varepsilon_t,\varepsilon_t) \\ &= \text{Cov}(\sum_{i=1}^p \varphi_i X_{t-i} + \sum_{i=1}^q \theta_i \varepsilon_{t-i},\varepsilon_t) + \text{Cov}(\varepsilon_t,\varepsilon_t) \\ &= 0 + \sigma^2 \\ &> 0. \end{aligned} $X_t$ and $\varepsilon_{t-j}$ are also dependent. Rewrite ARMA as MA($\infty$), and you will find that $X_t$ is a linear function of $\varepsilon_{t-j}$, and thus an analogous argument to the one above holds also here.
  2. No, because of 1.
  3. No, because of 1.

If you prefer empirical results to analytical proofs, simulate an ARMA process and check the empirical cross-correlation function of $X_t$ vs. $\varepsilon_t$. Here is how to do that in R:

n=1e6 # length of time series
set.seed(1)
eps=rnorm(n)
x=arima.sim(n=n, list(ar=c(0.8897,-0.4858), ma=c(-0.2279, 0.2488)), sd=1, innov=eps)
ccf(x,eps)

enter image description here

Improve this answer
$\endgroup$
6
  • $\begingroup$ Thank you for the answer. In this case, the answer to this question is wrong: stats.stackexchange.com/questions/196994/… To show the variance of $r_t=\alpha + \phi_1 r_{t-1} + \theta_1 a_{t-1} + a_{t}$, he makes use of the fact that $E(a_t r_{t-1})=0$. $\endgroup$
    – Fam
    Commented Dec 29, 2021 at 20:09
  • $\begingroup$ @Fam, that answer is not wrong, since it is $r$ that is lagged w.r.t. $a$ and not the other way around. You can observe this visually from the CCF graph I have included, or you could show it algebraically. $\endgroup$
    – Richard Hardy
    Commented Dec 29, 2021 at 20:36
  • $\begingroup$ So, $\varepsilon_t$ and $X_{t-1}$ are not correlated, in my question? Suppose that in the question in my first comment, we have $\alpha =0$. So, $E(r_t)=0$ and $E(r_{t-1})=0$. Thus, $cov(a_t,r_{t-1} ) = E(a_t r_{t-1})=0$. $\endgroup$
    – Fam
    Commented Dec 29, 2021 at 20:48
  • $\begingroup$ I dont underestand why $X_t$ and $\varepsilon_{t-1}$ are positive correlated, but $X_{t-1}$ and $\varepsilon_t$ are not? $\endgroup$
    – Fam
    Commented Dec 29, 2021 at 20:50
  • 1
    $\begingroup$ @Fam, your intuition is correct. $\varepsilon_t$ is generated independently of past values of $X$, but $X_t$ is a linear function of past values of $\varepsilon$. $\endgroup$
    – Richard Hardy
    Commented Dec 29, 2021 at 21:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.