# understanding time-homogeneous markov chain

Could anyone help me understand the definition on page 7 definition 2.25 here? I do not understand the notation $$(P(a))(A)$$ - what does this mean?

Also, is $$P(a, A)$$ a probability measure from the set of all Borel sets of $$X$$ to $$[0,1]$$?

I believe that $$P$$ is a function from $$X \times B(X)\to [0,1]$$?

Is this true?

Going strictly by what Definition 2.25 in the link says, $$P$$ is a function $$P : \mathcal{X} \to \mathcal{P}(\mathcal{X})$$, where $$\mathcal{P}(\mathcal{X})$$ is the space of probability measures on $$(\mathcal{X}, \mathscr{B}(\mathcal{X}))$$ (presumably $$\mathcal{X}$$ is a topological space and $$\mathscr{B}(\mathcal{X})$$ is the $$\sigma$$-algebra of Borel subsets of $$\mathcal{X}$$, but I didn't read the notes to check whether that's true). That means that for each $$a \in \mathcal{X}$$, $$P(a)$$ is a probability measure on $$(\mathcal{X}, \mathscr{B}(\mathcal{X}))$$, so for each $$A \in \mathscr{B}(\mathcal{X})$$, $$(P(a))(A)$$ is a number between $$0$$ and $$1$$.

You can think of $$P$$ as a "random probability measure" if you equip $$\mathcal{P}(\mathcal{X})$$ with a $$\sigma$$-algebra that makes $$P$$ a measurable function. Usually this is done by endowing $$\mathcal{P}(\mathcal{X})$$ with the $$\sigma$$-algebra $$\mathcal{H}$$ which is the smallest $$\sigma$$-algebra such that for each $$B \in \mathscr{B}(\mathcal{X})$$, the map $$Q \mapsto Q(B)$$ from $$\mathcal{P}(\mathcal{X})$$ to $$[0, 1]$$ is measurable (i.e., $$\mathcal{H}$$ is generated by all such maps). This way, $$P : \mathcal{X} \to \mathcal{P}(\mathcal{X})$$ is a measurable function, hence a "random element" of $$\mathcal{P}(\mathcal{X})$$.

The random probability measure approach is one way of defining the transition probability $$P$$. Another approach is using so-called transition kernels (AKA Markov kernels, etc.).

If $$(\mathcal{X}, \mathcal{F})$$ and $$(\mathcal{Y}, \mathcal{G})$$ are two measure spaces, then a transition kernel from $$(\mathcal{X}, \mathcal{F})$$ to $$(\mathcal{Y}, \mathcal{G})$$ is a function $$Q : \mathcal{X} \times \mathcal{G} \to [0, 1]$$ such that

1. for each $$x \in \mathcal{X}$$, the map $$Q(x, \cdot) : \mathcal{G} \to [0, 1]$$ is a probability measure on $$(\mathcal{Y}, \mathcal{G})$$, and
2. for each $$G \in \mathcal{G}$$, the map $$Q(\cdot, G) : \mathcal{X} \to [0, 1]$$ is $$\mathcal{F}$$-measurable.

Given a transition kernel $$Q$$ from $$(\mathcal{X}, \mathcal{F})$$ to $$(\mathcal{Y}, \mathcal{G})$$, we can define a random measure $$P : \mathcal{X} \to \mathcal{P}(\mathcal{Y})$$, where $$\mathcal{P}(\mathcal{Y})$$ stands for the space of probability measures on $$(\mathcal{Y}, \mathcal{G})$$, by defining $$(P(a)(A)) = Q(a, A)$$ for each $$a \in \mathcal{X}$$ and $$A \in \mathcal{G}$$. Conversely, given a random measure $$P : \mathcal{X} \to \mathcal{P}(\mathcal{Y})$$, we can define a transition kernel $$Q$$ from $$(\mathcal{X}, \mathcal{F})$$ to $$(\mathcal{Y}, \mathcal{G})$$ by the same relation. Thus, there is a one-to-one correspondence between random probability measures and transition kernels.

Because of this, it is common to use notation like $$P(a, A)$$ when $$P$$ is a random probability measure to stand for $$(P(a))(A)$$; the notation $$P(a, A)$$ is easier to read, and you're not really doing anything wrong by treating the random probability measure as a transition kernel.

In homogeneous markov chain the system dynamics do not change over time, I think it would be just the expression of the transition kernel which is fixed with time.

Maybe the typing in this textbook would be more expressive:

• Thanks for your answer, actually I am interested in understanding the notations and definition from the Hairer notes only, for my context. – riemann May 24 at 9:45