Why is $Z_t$ uncorrelated with $X_{t-1}$ in $X_t=\theta X_{t-1}+Z_t$? In a solution to the problem below, the teaching assistant solves it by calculating $\mathbb{E}[X_t^2]$ and ends up with also having to calculate $\mathbb{E}[X_{t-1}Z_t]$ after expanding the square. To do this, he states that $\mathbb{E}[X_{t-1}Z_t]=0$ since $X_{t-1}$ is independent of $Z_t$ and $X_{t-1}$ is uncorrelated with $Z_t$.
Questions:

*

*Why is $X_{t-1}$ independent of $Z_t$? I don't see that we assume that the TS is casual.

*Why is $X_{t-1}$ uncorrelated with $Z_t$?

Problem:

Let a timeseries model $X:=(X_t, t\in\mathbb{Z})$ be given by
$$X_t=\phi X_{t-1}+Z_t, \quad \text{where} \quad Z_t\sim
 \text{WN}(0,\sigma^2)\quad \text{and} \quad |\phi|\neq 1.$$
Assume that the stochastic process satisfying this model is stationary. Compute the variance of $X.$

Note: Yes, I know that one simply can calculate $\text{Var}[X_t]$ directly in one line, but I'm trying to understand the motivations behind the instructors steps.
 A: The general meaning of white noise (that's what that WN denotes) is that the  random variables $Z_t$ are independent. Some statisticians claim that it suffices to assume that the $Z_t$'s are uncorrelated (zero-mean, finite variance $\sigma^2$) random variables, that is,
$$\operatorname{cov}(Z_t, Z_{t^\prime}) = \begin{cases}\sigma^2, & t = t^\prime,\\0, & t \neq t^\prime\end{cases}\tag{1}$$
but some others (including your teaching assistant and non-statisticians such as myself) prefer the stronger requirement that the $Z_t$'s be independent (zero-mean, finite variance $\sigma^2$) random variables; i.i.d. random variables if you are familiar with the acronym. Of course, $(1)$ still holds when the $Z_t$'s are independent.  As your TA says, $X_{t-1}$ is independent of $Z_t$.
Actually, if you iterate the defining equation to write
\begin{align}
X_t &= \phi X_{t-1} + Z_t\\
&= \phi \big(\phi X_{t-2} + Z_{t-1}\big)  + Z_t\\
&= \phi^2 X_{t-2} + \phi Z_{t-1}  + Z_t\\
&= \phi^2 \big(\phi X_{t-3}  + Z_{t-2}\big)+ \phi Z_{t-1}  + Z_t\\
&= \phi^3 X_{t-3} + \phi^2 Z_{t-2} + \phi Z_{t-1}  + Z_t
\end{align}
and so on, you can use induction to deduce that
$X_t = \sum_{n=0}^\infty \phi^n Z_{t-n}$ and so , since the $Z_{t-n}$'s are independent random variables,
$$\operatorname{var}(Z_t) = \sum_{n=0}^\infty \operatorname{var}(\phi^nZ_{t-n}) = \sum_{n=0}^\infty \phi^{2n} \operatorname{var}(Z_{t-n}) = \sum_{n=0}^\infty \phi^{2n} \sigma^2 =\frac{\sigma^2}{1-\phi^2} \tag{2}$$
provided that $|\phi|<1$.  If $|\phi| > 1$, that series in $(2)$ diverges and it is not possible to write the value of $\operatorname{var}(Z_t)$ in the form shown on the right side of $(2)$. To answer a secondary question raised by the OP in a comment, it is not necessary to define a $X_0$ separately; the problem statement says that $t \in \mathbb Z$ and so $Z_t$ is defined for all integers $t$. Note that your TA's claim that $X_{t-1}$ is independent of $Z_t$ is perfectly valid: $X_{t-1}$ is a weighted sum of $Z_{t-1}, Z_{t-2}, \ldots$ and is thus independent of $Z_t$ by definition (uncorrelated with $Z_t$ for the naysayers).
