# proof of Strong Markov property check

### Statement

We say $$\Phi$$ has the strong Markov Property if for any initial distribution $$\mu$$, any real-valued bounded measurable function $$h$$ on $$\Omega$$, and any stopping time $$\zeta$$, $$\mathsf{E}_{\mu}[\theta^\zeta H \mid \mathcal{F}_\zeta^\Phi] = \mathsf{E}_{\Phi_\zeta}[H] \hspace{10mm} \mathsf{P}_\mu \text{-a.s.}$$ on the set $$\{\zeta < \infty\}$$.

### Proof

\begin{align} \mathsf{E}_{\mu}[\theta^\zeta H \mid \mathcal{F}_\zeta^\Phi] &= \mathsf{E}_{\mu}[\theta^\zeta H \sum_{k \ge 1} \mathbb{I}(\zeta = k) \mid \mathcal{F}_\zeta^\Phi] \tag{mult by 1}\\ &= \sum_{k \ge 1} \mathsf{E}_{\mu}[\theta^\zeta H \mathbb{I}(\zeta = k) \mid \mathcal{F}_\zeta^\Phi] \tag{linearity} \\ &= \sum_{k \ge 1} \mathsf{E}_{\mu}[\theta^k H \mathbb{I}(\zeta = k) \mid \mathcal{F}_\zeta^\Phi] \tag{\theta^\zeta = \theta^k on \{\zeta = k\} by defn.} \\ &= \sum_{k \ge 1} \mathbb{I}(\zeta = k) \mathsf{E}_{\mu}[\theta^k H \mid \mathcal{F}_\zeta^\Phi] \tag{indicator is \mathcal{F}_\zeta^\Phi-measurable} \\ &= \sum_{k \ge 1} \mathbb{I}(\zeta = k) \mathsf{E}_{\mu}[\theta^k H \mid \mathcal{F}_k^\Phi] \tag{?} \\ &= \sum_{k \ge 1} \mathbb{I}(\zeta = k) \mathsf{E}_{\Phi_k}[ H] \tag{(the other) Markov property} \\ &= \sum_{k \ge 1} \mathbb{I}(\zeta = k) \mathsf{E}_{\Phi_\zeta}[ H] \tag{?} \\ &= \mathsf{E}_{\Phi_\zeta}[ \sum_{k \ge 1} \mathbb{I}(\zeta = k) H] \\ &= \mathsf{E}_{\Phi_\zeta}[H] \end{align}

### Definitions

• $$\Phi = \{\Phi_0, \Phi_1, \ldots\}$$ is the chain defined on $$\Omega = \mathsf{X}^{\infty}$$
• $$\mu$$ is the chain's initial distribution on $$(\mathsf{X}, \mathcal{X})$$
• $$\mathsf{E}_\mu$$ and $$\mathsf{P}_\mu$$ are the expectation and probability measure defined on the entire product measurable space $$(\Omega, \mathcal{F})$$
• $$\zeta$$ is the stopping time
• $$\mathcal{F}_{\zeta}^\Phi$$ is the sigma-field that describes all the events that happen up to time $$\zeta$$
• $$\theta^{\zeta} : \{x_0, x_1, \ldots\} \mapsto \{x_{\zeta}, x_{\zeta+1}, \ldots\}$$ is the (random) shift operator
• $$H = h(\Phi_0, \Phi_1, \ldots)$$ is a random variable made from the bounded measurable function $$h$$
• $$\theta^\zeta H = H \circ \theta^\zeta(\omega)$$ is the function $$h$$ applied to randomly-delayed chain

I can do $$\mathbb{I}(\zeta = k) \mathsf{E}_{\mu}[\theta^k H \mid \mathcal{F}_\zeta^\Phi] = \mathbb{I}(\zeta = k) \mathsf{E}_{\mu}[\theta^k H \mid \mathcal{F}_k^\Phi]$$ because

$$\int_A \mathbb{I}(\zeta = k) \mathsf{E}_{\mu}[\theta^k H \mid \mathcal{F}_\zeta^\Phi] \mathsf{P}_\mu(d\omega) = \int_A \mathbb{I}(\zeta = k) \mathsf{E}_{\mu}[\theta^k H \mid \mathcal{F}_k^\Phi] \mathsf{P}_\mu(d\omega) \tag{*}$$ for any $$A \in \mathcal{F}_\zeta^\Phi$$, right? By definition of $$\mathcal{F}_\zeta^\Phi$$ $$A \cap \{\zeta = k\} \in \mathcal{F}_k$$ so $$\mathbb{I}(\omega \in A)\mathbb{I}(\zeta = k)$$ is $$\mathcal{F}_k$$-measurable. This gives us that both sides of $$(*)$$ equal $$\mathsf{E}_{\mu}[\mathbb{I}(\omega \in A)\theta^k H ]$$.

• You write $$\mathsf{E}_{\mu}[\theta^\zeta H \sum_{k \ge 1} \mathbb{I}(\zeta = k) \mid \mathcal{F}_\zeta^\Phi]$$ but what is $\zeta$ to which $k$ can be compared in the indicator function? Nov 17, 2020 at 13:09
• @SextusEmpiricus it's the stopping time. I'm doing the thing where I partition up $\Omega$ and multiply by $\mathbb{I}(A) + \mathbb{I}(A^c)$ Nov 17, 2020 at 14:49
• But isn't the stopping time multivalued? For instance, if you have a Brownian motion and consider the stopping time the time when the particle hits some boundary or passes some level, what is then $\mathbb{I}(\zeta = k)$? Which value do you put in? Nov 17, 2020 at 15:08
• @SextusEmpiricus this is a discrete-time Markov chain, and so this is a countable sum. Random variables that take into account stopping times are defined piecewise on individual sets. On the set $\{\zeta = k\} \subseteq \Omega$, $\zeta$ evaluates to $k$ Nov 17, 2020 at 15:48
• So you are looking for the property $\sum_{k} \delta_{k,\zeta} \mathsf{E}_{\mu}[\theta^\zeta H \mid \mathcal{F}_\zeta^\Phi] = \mathsf{E}_{\mu}[\theta^k H \mid \mathcal{F}_k^\Phi]$ with $\delta_{k,\zeta}$ the Kronecker delta. Nov 17, 2020 at 15:55

The two question marked steps are indeed correct. This proof also follows the proof given for Proposition 3.4.6 here. Note that the assumption of $$P(\tau = \infty) = 0$$ comes into play when we sum over $$1,2,\ldots$$.