3
$\begingroup$

Let $X_n=\sum\limits_{k=1}^n Y_k = X_{n-1} + Y_n$ and $X_0=0$, $(Y_n)$ i.i.d with

$P(Y_n=1)=p=1-P(Y_n=-1)=1-q, p \in (0,1)$, $(X_n)$ is a random walk on $\mathbb{Z}$.

Why is this Markov chain irreducible? How do we know that all states (integers) communicate with each other? How do we show this intuitively?

$\endgroup$
1
  • $\begingroup$ Hint: What's the probability you're at state $i$ after exactly $i$ steps? How about that of $-i$ at step $i$? What must happen regarding other states visited in order to get there? Conclusion? $\endgroup$
    – cardinal
    Commented Jan 28, 2013 at 2:01

1 Answer 1

5
$\begingroup$

Two states $x,y$ communicate if it is possible to get from $x$ to $y$ with positive probability and viceversa. If all the states communicate then the chain is said to be irreducible.

The chain you described is irreducible because there is always a non-zero probability to go from every state $x \in \mathbb{Z}$ to another $y \in \mathbb{Z}$ in a finite number of steps. In particular, from $x$ you can get to $y$ in $y-x$ steps with probability $p^{(y-x)}$ if $x \leq y$ or with probability $(1-p)^{(y-x)}$ if $x > y$.

You can think of a random walk as a process that takes steps by tossing a biased coin: a Head corresponds to moving up, a Tail corresponds to moving down. There is always a non-zero probability of obtaining enough Heads (or Tails) in a row such that you can reach any target position from any initial position.

$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.