# Questions tagged [transition-matrix]

A transition matrix is a square matrix used to describe the transitions of a Markov chain.

101 questions
Filter by
Sorted by
Tagged with
1 vote
18 views

### Markov chain: inferring transition rates from equilibrium

I feel like I have been having a very dumb week trying to solve/research this problem and that I am missing an easy solution or that it is not possible. Given an equilibrium distribution (say from a ...
1 vote
21 views

### How to prove a sequence of random variables dependent on different random variables is a homogeneous Markov chain?

In Pierre Brémaud's book, Markov Chains - Gibbs Fields, Monte Carlo Simulation and Queues, exercise 2.6.9 is stated as follows: Let $\{Z_n\}_{n \geq 1}$ be an IID sequence of geometric random ...
• 648
1 vote
90 views

### Coin flipping (Markov chain)

Suppose that coin 1 has probability 0.7 of coming up heads, and coin 2 has probability 0.6 of coming up heads. If the coin flipped today comes up heads, then we select coin 1 to flip tomorrow, and if ...
• 1,118
19 views

### How to decide $X_n, n \in [0,\infty)$ is a Markov chain and how to compute transition probability matrix?

Three white and three black balls are distributed in two urns in such a way that each contains three balls. We say that the system is in state i, i = 0, 1, 2, 3, if the first urn contains i white ...
• 1,118
29 views

• 1,400
208 views

### How to show that the transition probability is equal to $\overline p_{ij} = \frac{P_{ij}}{\sum_{k\neq i}p_{ik}}$

(No new answers needed) I would like to award @whuber for his good answer with my bounty! Suppose that $(X_n)_{n≥0}$ is Markov$(λ, P)$ but that we only observe the process when it moves to a new ... 1 vote
813 views

### Rows and columns of the one-step transition probability matrix

I am currently studying the textbook Introduction to Modeling and Analysis of Stochastic Systems, Second Edition, by V. G. Kulkarni. In a section on discrete-time Markov chains, the author introduces ...
• 1,186
65 views

### Singular state transition probability matrix in David Silver's UCL Lesson 2

I'm studying David Silver's second lesson on reinforcement learning: https://www.youtube.com/watch?v=lfHX2hHRMVQ&list=PLqYmG7hTraZDM-OYHWgPebj2MfCFzFObQ&index=2 and the state transition ...
1 vote
23 views

### Specifying a multi-state model with unobservable terminal state

Suppose there is a multi-state process with three states, listed below and labelled as terminal/non-terminal and observable/unobservable: Initializing: non-terminal observable Active: non-terminal ...
• 3,683
219 views

### Make sense of plotting a transition matrix

I'm studying statistics and I'm trying to understand markov chain topic. I'm using the package "markovchain" in R to obtain the stationary distribution. From this transition matrix $M$: <...
862 views

### Measure the distance between two probability transition matrices

I have a probability transition matrix $P$ that contains some values very close to zero. I want to sparsify this matrix by taking the k largest values for each row and setting the others to zero. For ...
73 views

### Long run proportion of transitions in a Markov chain

Let $S$ be a set of states for a Markov chain and let $S^C$ be the remaining states. Explain the identity $$\sum_{i\in S}\sum_{j\in S^C}\pi_iP_{ij}=\sum_{i\in S^C}\sum_{j\in S}\pi_iP_{ij}$$ I know ...
1 vote
491 views

### Irreducible (communicating) classes [closed]

The Markov chain $(Xn; n\geq)$ has state-space $S = (0, 1, 2, . . .)$, with $p_{i,0} = \frac{1}{4}$ and $p_{i,i+1} = \frac{3}{4}$ $\forall i \geq 0$, so that the transition matrix is P =\$\...
1 vote
548 views

### Biased coins and Markov processes

Good day, I am attempting an optional exercise and I am finding it hard to interpret the problem in terms of matrices and vectors. Coin 1 has probability 0.4 of coming up heads, and coin 2 has ...
112 views

### Formulating a Transition matrix for Markov Process

I am dealing with a medical process which is as follows. There are 10000 Veterans who are enrolled in this study. All 10000 have medical condition called onychocryptosis which is a fancy term for ...
• 375