# Markov chain and process

This is not homework. Just practicing for an upcoming exam. Question is taken from a web pdf : http://www.dartmouth.edu/~chance/teaching_aids/books_articles/probability_book/Chapter11.pdf

Consider the following process. We have two coins, one of which is fair, and the other of which has heads on both sides. We give these two coins to our friend, who chooses one of them at random (each with probability 1/2). During the rest of the process, she uses only the coin that she chose. She now proceeds to toss the coin many times, reporting the results. We consider this process to consist solely of what she reports to us.

(a) Given that she reports a head on the nth toss, what is the probability that a head is thrown on the (n + 1)st toss?

(b) Consider this process as having two states, heads and tails. By computing the other three transition probabilities analogous to the one in part (a), write down a \transition matrix" for this process.

(c) Now assume that the process is in state "heads" on both the (n - 1)st and the nth toss. Find the probability that a head comes up on the (n + 1)st toss.

(d) Is this process a Markov chain?

Solution is given here: Chapter 11 question 19 :http://www.dartmouth.edu/~chance/teaching_aids/books_articles/probability_book/Answersodd-10-14-08.pdf

Also, what is the best approach in assigning state i a state? "state 1=..."? Also, what is the difference between the Markov process and Markov chain?

• You should add the self-study tag, since this is for "[a] routine question from a textbook, course, or test used for a class or self-study." – Patrick Coulombe Feb 25 '14 at 3:22
• Easily solved from using the probability tree. It is not a Markov process because the answer for part c will equal to part a as Markhov is dependent on the i and i-1 process. I cannot place my answer as I don't have the rep to. – user40837 Feb 25 '14 at 4:08
• The privilege to post answers seems to be 1 rep. You should be able to post answers. It might have been a new user time limit (trying to answer too quickly), perhaps. – Glen_b -Reinstate Monica Feb 25 '14 at 5:17