0
$\begingroup$

Wiki gives this definition of a discrete-time Markov chain

a sequence of random variables $X_1$, $X_2$, $X_3$, ... with the Markov property, namely that the probability of moving to the next state depends only on the present state and not on the previous states:

$\Pr(X_{n+1}=x\mid X_{1}=x_{1},X_{2}=x_{2},\ldots ,X_{n}=x_{n})=\Pr(X_{n+1}=x\mid X_{n}=x_{n}),$ if both conditional probabilities are well defined, that is, if

$\Pr(X_{1}=x_{1},\ldots ,X_{n}=x_{n})>0.$

The possible values of $X_i$ form a countable set S called the state space of the chain.

Wiki also uses this example to illustrate Markov chains.

The probabilities of weather conditions (modeled as either rainy or sunny), given the weather on the preceding day, can be represented by a transition matrix:

${\displaystyle P={\begin{bmatrix}0.9&0.1\\0.5&0.5\end{bmatrix}}}$

The matrix P represents the weather model in which a sunny day is 90% likely to be followed by another sunny day, and a rainy day is 50% likely to be followed by another rainy day. The columns can be labelled "sunny" and "rainy", and the rows can be labelled in the same order.

The weather on day 1 is known to be sunny. This is represented by a vector in which the "sunny" entry is 100%, and the "rainy" entry is 0%:

${\displaystyle \mathbf {x} ^{(0)}={\begin{bmatrix}1&0\end{bmatrix}}}$

for day n + 1(Note: the original value on wiki is n, which seems to be incorrect)

${\mathbf {x}}^{{(n)}}={\mathbf {x}}^{{(0)}}P^{n}$

The superscript (n) is an index, and not an exponent.

In the particular case, the state space of the chain is {rainy , sunny}

so, what is the minimum of n?

in particular, ${\displaystyle \Pr(X_0=sunny) = 1,}$ is this a Markov chain?

${\displaystyle \Pr(X_{1}=sunny\mid X_0=sunny)}$, is this a Markov chain?

${\displaystyle \Pr(X_{2}=sunny\mid X_0=sunny,X_{1}=sunny)}$ is this a Markov chain?

$\endgroup$
3
$\begingroup$

A Markov chain is a stochastic process.

A stochastic process in discrete time n ∈ $N$ = {0, 1, 2, . . .} is a sequence of random variables (rvs) $X_0, X_1, X_2$, . . . denoted by $X = \{X_n : n ≥ 0\}$.

So, for your question, each of them is a Markov chain.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.