We say $\Phi$ has the strong Markov Property if for any initial distribution $\mu$, any real-valued bounded measurable function $h$ on $\Omega$, and any stopping time $\zeta$, $$ \mathsf{E}_{\mu}[\theta^\zeta H \mid \mathcal{F}_\zeta^\Phi] = \mathsf{E}_{\Phi_\zeta}[H] \hspace{10mm} \mathsf{P}_\mu \text{-a.s.} $$ on the set $\{\zeta < \infty\}$.


\begin{align} \mathsf{E}_{\mu}[\theta^\zeta H \mid \mathcal{F}_\zeta^\Phi] &= \mathsf{E}_{\mu}[\theta^\zeta H \sum_{k \ge 1} \mathbb{I}(\zeta = k) \mid \mathcal{F}_\zeta^\Phi] \tag{mult by $1$}\\ &= \sum_{k \ge 1} \mathsf{E}_{\mu}[\theta^\zeta H \mathbb{I}(\zeta = k) \mid \mathcal{F}_\zeta^\Phi] \tag{linearity} \\ &= \sum_{k \ge 1} \mathsf{E}_{\mu}[\theta^k H \mathbb{I}(\zeta = k) \mid \mathcal{F}_\zeta^\Phi] \tag{$\theta^\zeta = \theta^k$ on $\{\zeta = k\}$ by defn.} \\ &= \sum_{k \ge 1} \mathbb{I}(\zeta = k) \mathsf{E}_{\mu}[\theta^k H \mid \mathcal{F}_\zeta^\Phi] \tag{indicator is $\mathcal{F}_\zeta^\Phi$-measurable} \\ &= \sum_{k \ge 1} \mathbb{I}(\zeta = k) \mathsf{E}_{\mu}[\theta^k H \mid \mathcal{F}_k^\Phi] \tag{?} \\ &= \sum_{k \ge 1} \mathbb{I}(\zeta = k) \mathsf{E}_{\Phi_k}[ H] \tag{(the other) Markov property} \\ &= \sum_{k \ge 1} \mathbb{I}(\zeta = k) \mathsf{E}_{\Phi_\zeta}[ H] \tag{?} \\ &= \mathsf{E}_{\Phi_\zeta}[ \sum_{k \ge 1} \mathbb{I}(\zeta = k) H] \\ &= \mathsf{E}_{\Phi_\zeta}[H] \end{align}


  • $\Phi = \{\Phi_0, \Phi_1, \ldots\}$ is the chain defined on $\Omega = \mathsf{X}^{\infty}$
  • $\mu$ is the chain's initial distribution on $(\mathsf{X}, \mathcal{X})$
  • $\mathsf{E}_\mu$ and $\mathsf{P}_\mu$ are the expectation and probability measure defined on the entire product measurable space $(\Omega, \mathcal{F})$
  • $\zeta$ is the stopping time
  • $\mathcal{F}_{\zeta}^\Phi$ is the sigma-field that describes all the events that happen up to time $\zeta$
  • $\theta^{\zeta} : \{x_0, x_1, \ldots\} \mapsto \{x_{\zeta}, x_{\zeta+1}, \ldots\}$ is the (random) shift operator
  • $H = h(\Phi_0, \Phi_1, \ldots)$ is a random variable made from the bounded measurable function $h$
  • $\theta^\zeta H = H \circ \theta^\zeta(\omega)$ is the function $h$ applied to randomly-delayed chain

Questions And Comments

I can do $$ \mathbb{I}(\zeta = k) \mathsf{E}_{\mu}[\theta^k H \mid \mathcal{F}_\zeta^\Phi] = \mathbb{I}(\zeta = k) \mathsf{E}_{\mu}[\theta^k H \mid \mathcal{F}_k^\Phi] $$ because

$$ \int_A \mathbb{I}(\zeta = k) \mathsf{E}_{\mu}[\theta^k H \mid \mathcal{F}_\zeta^\Phi] \mathsf{P}_\mu(d\omega) = \int_A \mathbb{I}(\zeta = k) \mathsf{E}_{\mu}[\theta^k H \mid \mathcal{F}_k^\Phi] \mathsf{P}_\mu(d\omega) \tag{*} $$ for any $A \in \mathcal{F}_\zeta^\Phi$, right? By definition of $\mathcal{F}_\zeta^\Phi$ $$ A \cap \{\zeta = k\} \in \mathcal{F}_k $$ so $\mathbb{I}(\omega \in A)\mathbb{I}(\zeta = k)$ is $\mathcal{F}_k$-measurable. This gives us that both sides of $(*)$ equal $\mathsf{E}_{\mu}[\mathbb{I}(\omega \in A)\theta^k H ]$.

  • $\begingroup$ You write $$ \mathsf{E}_{\mu}[\theta^\zeta H \sum_{k \ge 1} \mathbb{I}(\zeta = k) \mid \mathcal{F}_\zeta^\Phi] $$ but what is $\zeta$ to which $k$ can be compared in the indicator function? $\endgroup$ Commented Nov 17, 2020 at 13:09
  • $\begingroup$ @SextusEmpiricus it's the stopping time. I'm doing the thing where I partition up $\Omega$ and multiply by $\mathbb{I}(A) + \mathbb{I}(A^c)$ $\endgroup$
    – Taylor
    Commented Nov 17, 2020 at 14:49
  • $\begingroup$ But isn't the stopping time multivalued? For instance, if you have a Brownian motion and consider the stopping time the time when the particle hits some boundary or passes some level, what is then $\mathbb{I}(\zeta = k)$? Which value do you put in? $\endgroup$ Commented Nov 17, 2020 at 15:08
  • $\begingroup$ @SextusEmpiricus this is a discrete-time Markov chain, and so this is a countable sum. Random variables that take into account stopping times are defined piecewise on individual sets. On the set $\{\zeta = k\} \subseteq \Omega$, $\zeta$ evaluates to $k$ $\endgroup$
    – Taylor
    Commented Nov 17, 2020 at 15:48
  • $\begingroup$ So you are looking for the property $\sum_{k} \delta_{k,\zeta} \mathsf{E}_{\mu}[\theta^\zeta H \mid \mathcal{F}_\zeta^\Phi] = \mathsf{E}_{\mu}[\theta^k H \mid \mathcal{F}_k^\Phi]$ with $\delta_{k,\zeta}$ the Kronecker delta. $\endgroup$ Commented Nov 17, 2020 at 15:55

1 Answer 1


The two question marked steps are indeed correct. This proof also follows the proof given for Proposition 3.4.6 here. Note that the assumption of $P(\tau = \infty) = 0$ comes into play when we sum over $1,2,\ldots$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.