4
$\begingroup$

In a solution to the problem below, the teaching assistant solves it by calculating $\mathbb{E}[X_t^2]$ and ends up with also having to calculate $\mathbb{E}[X_{t-1}Z_t]$ after expanding the square. To do this, he states that $\mathbb{E}[X_{t-1}Z_t]=0$ since $X_{t-1}$ is independent of $Z_t$ and $X_{t-1}$ is uncorrelated with $Z_t$.

Questions:

  1. Why is $X_{t-1}$ independent of $Z_t$? I don't see that we assume that the TS is casual.
  2. Why is $X_{t-1}$ uncorrelated with $Z_t$?

Problem:

Let a timeseries model $X:=(X_t, t\in\mathbb{Z})$ be given by

$$X_t=\phi X_{t-1}+Z_t, \quad \text{where} \quad Z_t\sim \text{WN}(0,\sigma^2)\quad \text{and} \quad |\phi|\neq 1.$$

Assume that the stochastic process satisfying this model is stationary. Compute the variance of $X.$

Note: Yes, I know that one simply can calculate $\text{Var}[X_t]$ directly in one line, but I'm trying to understand the motivations behind the instructors steps.

$\endgroup$
10
  • 4
    $\begingroup$ I suspect that $Z_t$ is assumed to be i.i.d., and independent of $X_t$ (and therefore also of the other $X_s, s< t$ in case this holds for $X_0$) without this being explicitly mentioned. $\endgroup$ Aug 19, 2021 at 13:58
  • 2
    $\begingroup$ Q2: Independent $\Rightarrow$ uncorrelated. $\endgroup$ Aug 19, 2021 at 14:00
  • 1
    $\begingroup$ To get $\mathbb{E}[X_{t-1}Z_t]=0$, in addition to independent/uncorrelated you need $\mathbb{E}[X_{t-1}]=0$ or $\mathbb{E}[Z_t]=0$ $\endgroup$
    – Henry
    Aug 19, 2021 at 14:18
  • 2
    $\begingroup$ @Henry $Z_t \sim WN(0,\sigma^2)$ seems to allow the assumption that $E[Z_t] = 0.$ $\endgroup$ Aug 19, 2021 at 14:42
  • 1
    $\begingroup$ I think you need $|\phi| < 1$, not $|\phi| \neq 1$, in your problem statement. $\endgroup$ Aug 19, 2021 at 14:49

1 Answer 1

10
$\begingroup$

The general meaning of white noise (that's what that WN denotes) is that the random variables $Z_t$ are independent. Some statisticians claim that it suffices to assume that the $Z_t$'s are uncorrelated (zero-mean, finite variance $\sigma^2$) random variables, that is, $$\operatorname{cov}(Z_t, Z_{t^\prime}) = \begin{cases}\sigma^2, & t = t^\prime,\\0, & t \neq t^\prime\end{cases}\tag{1}$$ but some others (including your teaching assistant and non-statisticians such as myself) prefer the stronger requirement that the $Z_t$'s be independent (zero-mean, finite variance $\sigma^2$) random variables; i.i.d. random variables if you are familiar with the acronym. Of course, $(1)$ still holds when the $Z_t$'s are independent. As your TA says, $X_{t-1}$ is independent of $Z_t$.

Actually, if you iterate the defining equation to write \begin{align} X_t &= \phi X_{t-1} + Z_t\\ &= \phi \big(\phi X_{t-2} + Z_{t-1}\big) + Z_t\\ &= \phi^2 X_{t-2} + \phi Z_{t-1} + Z_t\\ &= \phi^2 \big(\phi X_{t-3} + Z_{t-2}\big)+ \phi Z_{t-1} + Z_t\\ &= \phi^3 X_{t-3} + \phi^2 Z_{t-2} + \phi Z_{t-1} + Z_t \end{align} and so on, you can use induction to deduce that $X_t = \sum_{n=0}^\infty \phi^n Z_{t-n}$ and so , since the $Z_{t-n}$'s are independent random variables, $$\operatorname{var}(Z_t) = \sum_{n=0}^\infty \operatorname{var}(\phi^nZ_{t-n}) = \sum_{n=0}^\infty \phi^{2n} \operatorname{var}(Z_{t-n}) = \sum_{n=0}^\infty \phi^{2n} \sigma^2 =\frac{\sigma^2}{1-\phi^2} \tag{2}$$ provided that $|\phi|<1$. If $|\phi| > 1$, that series in $(2)$ diverges and it is not possible to write the value of $\operatorname{var}(Z_t)$ in the form shown on the right side of $(2)$. To answer a secondary question raised by the OP in a comment, it is not necessary to define a $X_0$ separately; the problem statement says that $t \in \mathbb Z$ and so $Z_t$ is defined for all integers $t$. Note that your TA's claim that $X_{t-1}$ is independent of $Z_t$ is perfectly valid: $X_{t-1}$ is a weighted sum of $Z_{t-1}, Z_{t-2}, \ldots$ and is thus independent of $Z_t$ by definition (uncorrelated with $Z_t$ for the naysayers).

$\endgroup$
7
  • $\begingroup$ Probably as perfect of an explanation as I could have hoped for. Thank you kind sir! $\endgroup$
    – Parseval
    Aug 19, 2021 at 14:52
  • 1
    $\begingroup$ Some question: You say that $Z_t$ are independent of any other random variables in the system. Does this mean that $Z_t$ is independent of $X_t$ as well? Clearly the value of $X_t$ depends on $Z_t$ in the first equation in your array? Also, to make that recursion work, don't you need a value for $X_0?$ $\endgroup$
    – Parseval
    Aug 19, 2021 at 14:59
  • 1
    $\begingroup$ @Parseval $Z_t$ and $X_t$ are not independent; $Z_t$ and $X_{t-1}$ are independent. $\endgroup$ Aug 19, 2021 at 22:54
  • 1
    $\begingroup$ @Kevin Let $X\sim N(0,1)$ and $\{Z_n\}$ a collection of iid random variables taking on values $\pm 1$ with equal probability $\frac 12$, independent of $X$. Define $Y_n=XZ_n$. Then, $Y_n\sim N(0,1)$ and for $n\neq m$, $E[Y_nY_m]=E[X^2]E[Z_n]E[Z_m]=0$ and so the $Y_n$'s constitute a white (Gaussian) noise process according to the naysayers. But, given $Y_n=y$, we know that all the $Y_m$ necessarily have value $\pm y$ !! I think this is a shoddy model for white Gaussian noise, but ymmv. Why insist on hair-splitting between uncorrelated white noise and independent white noise which (continued) $\endgroup$ Aug 20, 2021 at 15:59
  • 1
    $\begingroup$ (continued) ... the cognoscenti on this forum prefer when just defining (discrete-time) white noise is a sequence of iid random variables with distributions symmetric about $0$ saves everyone a lot of grief? The uncorrelated white noise concept is usefully only for linear or OLS regression, but not for more general purposes, so why insist on it? $\endgroup$ Aug 20, 2021 at 16:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.