# Markov Chains and First Return Time

I'm having a difficult time with discrete Markov Chains. In particular, I need help with part c, d, and e on this problem. I do not understand how you can directly calculate the expected first return time to state 6 in part c, how to calculate the probability that the first return time to 6 is finite given you start in state 5, and the probability that the first return time to 6 is finite given you start in state 3. Any help would be great, thanks!

Here is a picture to the problem. By the way, P is the transition probability matrix for X where X is a Markov Chain with state space S={1,2,3,4,5,6}.

Hint: If you start in $\lbrace 4,5,6\rbrace$, you stay in $\lbrace 4,5,6\rbrace$, and you have a $1/3$ chance to move to $6$ at each step.