Statement
We say $\Phi$ has the strong Markov Property if for any initial distribution $\mu$, any real-valued bounded measurable function $h$ on $\Omega$, and any stopping time $\zeta$, $$ \mathsf{E}_{\mu}[\theta^\zeta H \mid \mathcal{F}_\zeta^\Phi] = \mathsf{E}_{\Phi_\zeta}[H] \hspace{10mm} \mathsf{P}_\mu \text{-a.s.} $$ on the set $\{\zeta < \infty\}$.
Proof
\begin{align} \mathsf{E}_{\mu}[\theta^\zeta H \mid \mathcal{F}_\zeta^\Phi] &= \mathsf{E}_{\mu}[\theta^\zeta H \sum_{k \ge 1} \mathbb{I}(\zeta = k) \mid \mathcal{F}_\zeta^\Phi] \tag{mult by $1$}\\ &= \sum_{k \ge 1} \mathsf{E}_{\mu}[\theta^\zeta H \mathbb{I}(\zeta = k) \mid \mathcal{F}_\zeta^\Phi] \tag{linearity} \\ &= \sum_{k \ge 1} \mathsf{E}_{\mu}[\theta^k H \mathbb{I}(\zeta = k) \mid \mathcal{F}_\zeta^\Phi] \tag{$\theta^\zeta = \theta^k$ on $\{\zeta = k\}$ by defn.} \\ &= \sum_{k \ge 1} \mathbb{I}(\zeta = k) \mathsf{E}_{\mu}[\theta^k H \mid \mathcal{F}_\zeta^\Phi] \tag{indicator is $\mathcal{F}_\zeta^\Phi$-measurable} \\ &= \sum_{k \ge 1} \mathbb{I}(\zeta = k) \mathsf{E}_{\mu}[\theta^k H \mid \mathcal{F}_k^\Phi] \tag{?} \\ &= \sum_{k \ge 1} \mathbb{I}(\zeta = k) \mathsf{E}_{\Phi_k}[ H] \tag{(the other) Markov property} \\ &= \sum_{k \ge 1} \mathbb{I}(\zeta = k) \mathsf{E}_{\Phi_\zeta}[ H] \tag{?} \\ &= \mathsf{E}_{\Phi_\zeta}[ \sum_{k \ge 1} \mathbb{I}(\zeta = k) H] \\ &= \mathsf{E}_{\Phi_\zeta}[H] \end{align}
Definitions
- $\Phi = \{\Phi_0, \Phi_1, \ldots\}$ is the chain defined on $\Omega = \mathsf{X}^{\infty}$
- $\mu$ is the chain's initial distribution on $(\mathsf{X}, \mathcal{X})$
- $\mathsf{E}_\mu$ and $\mathsf{P}_\mu$ are the expectation and probability measure defined on the entire product measurable space $(\Omega, \mathcal{F})$
- $\zeta$ is the stopping time
- $\mathcal{F}_{\zeta}^\Phi$ is the sigma-field that describes all the events that happen up to time $\zeta$
- $\theta^{\zeta} : \{x_0, x_1, \ldots\} \mapsto \{x_{\zeta}, x_{\zeta+1}, \ldots\}$ is the (random) shift operator
- $H = h(\Phi_0, \Phi_1, \ldots)$ is a random variable made from the bounded measurable function $h$
- $\theta^\zeta H = H \circ \theta^\zeta(\omega)$ is the function $h$ applied to randomly-delayed chain
Questions And Comments
I can do $$ \mathbb{I}(\zeta = k) \mathsf{E}_{\mu}[\theta^k H \mid \mathcal{F}_\zeta^\Phi] = \mathbb{I}(\zeta = k) \mathsf{E}_{\mu}[\theta^k H \mid \mathcal{F}_k^\Phi] $$ because
$$ \int_A \mathbb{I}(\zeta = k) \mathsf{E}_{\mu}[\theta^k H \mid \mathcal{F}_\zeta^\Phi] \mathsf{P}_\mu(d\omega) = \int_A \mathbb{I}(\zeta = k) \mathsf{E}_{\mu}[\theta^k H \mid \mathcal{F}_k^\Phi] \mathsf{P}_\mu(d\omega) \tag{*} $$ for any $A \in \mathcal{F}_\zeta^\Phi$, right? By definition of $\mathcal{F}_\zeta^\Phi$ $$ A \cap \{\zeta = k\} \in \mathcal{F}_k $$ so $\mathbb{I}(\omega \in A)\mathbb{I}(\zeta = k)$ is $\mathcal{F}_k$-measurable. This gives us that both sides of $(*)$ equal $\mathsf{E}_{\mu}[\mathbb{I}(\omega \in A)\theta^k H ]$.
