Questions tagged [transition-matrix]
A transition matrix is a square matrix used to describe the transitions of a Markov chain.
103
questions
0
votes
0
answers
24
views
stationary distribution of a continuous time markov chain
With a certain rate $R$ balls fall into a box. There is no limit to the number of balls the box can hold, but each ball has a rate $\gamma$ to leave the box and when two balls hit each other they ...
- 1
1
vote
0
answers
25
views
In Markov process, how to solve $P(X_n=1)$
Let $X_n$ is Markov chain for $n\geq 0$ and state space is $E=\{1,2,3\}$. One-step transition probability matrix is
$$
\left[ \begin{matrix}
p_{11}& p_{12}& p_{13}\\
p_{21}& p_{22}&...
- 11
1
vote
1
answer
40
views
Estimating Markov chain transition probabilities from data
In a discrete time and space Markov chain, I know the formula to estimate the transition probabilities $$p_{ij} = \frac{n_{ij}}{\sum_{j \in S} n_{ij}}$$ I'm not sure however how you can find this is ...
- 13
1
vote
1
answer
41
views
Markov chain: inferring transition rates from equilibrium
I feel like I have been having a very dumb week trying to solve/research this problem and that I am missing an easy solution or that it is not possible. Given an equilibrium distribution (say from a ...
- 11
1
vote
0
answers
79
views
How to prove a sequence of random variables dependent on different random variables is a homogeneous Markov chain?
In Pierre Brémaud's book, Markov Chains - Gibbs Fields, Monte Carlo Simulation and Queues, exercise 2.6.9 is stated as follows:
Let $\{Z_n\}_{n \geq 1}$ be an IID sequence of geometric random ...
- 610
1
vote
1
answer
479
views
Coin flipping (Markov chain)
Suppose that coin 1 has probability 0.7 of coming up heads, and coin 2 has probability 0.6 of coming up heads.
If the coin flipped today comes up heads, then we select coin 1 to flip tomorrow, and if ...
- 1,140
0
votes
0
answers
21
views
How to decide $X_n, n \in [0,\infty)$ is a Markov chain and how to compute transition probability matrix?
Three white and three black balls are distributed in two urns in such a way that each contains three balls. We say that the system is in state i, i = 0, 1, 2, 3, if the first urn contains i white ...
- 1,140
0
votes
0
answers
61
views
Is this transition probability matrix correct?
A pensioner receives 2 (thousand dollars) at the beginning of each month. The amount of money he needs to spend during a month is independent of the amount he has and is equal to i with probability $...
- 1,140
0
votes
0
answers
9
views
Interpreting Chart of Transition Densities from MSBVAR Gibbs Sampling
I've run the MSBVAR package on 3 time series, and then did a Gibbs Sampling and used the plotregimeid function based on the elements of the transition matrix Q. I'm trying to get an understanding of ...
- 1
0
votes
0
answers
15
views
When using Markov models for attribution modeling,what information does the transition matrix have,that causes the steady state vector not to be used?
I've just finished my msc thesis in attribution modeling, comparing higher order Markov models and the heuristic approaches. The professor's question is what information does the transition matrix ...
- 1
1
vote
0
answers
63
views
How to map a sequence to a transition matrix
I have the following transition matrices, one for Maria and one for Anna:
...
- 431
0
votes
0
answers
14
views
0
votes
1
answer
157
views
What are the transition functions for RNNs
From what I understand, the hidden states of RNNs are equivalent to the deterministic probability distribution over hidden states in for example a Hidden Markov Model.
Thus, just as probabilistic ...
- 135
0
votes
0
answers
93
views
R CODE Transition Matrix having zeroes for some transitions
I have been trying to create a Transition Matrix using the data from 2000 entities over 40 observations (Years). I have ranked the data into percentiles, for example the highest value entity in a ...
0
votes
0
answers
36
views
Estimating transition probabilities of a Markov chain with bayesian approach
I am trying to model heteroskedasticity in time series data,and the volatility $\left(\sigma_{t}\right)$ is taken as a Markov chain with two values $\sigma_{h}>\sigma_{l}>0$ with transition ...
- 3
1
vote
0
answers
78
views
Approximating a countable-state (infinite) Markov model with a finite-state one
TL;DR—in a nutshell
I have a countable-state Markov model (with a countably infinite number of states) in which the probability of transitioning to states $S_{i>k}$ for large $k$s are practically ...
- 301
1
vote
1
answer
156
views
R MSM: unexpected identical transition rate matrices for different values of covariate; why?
I am fitting a continuous-time Markov model to a panel dataset using the R package MSM. Because I am interested in sex-differences in transition rates, I fit the ...
- 1,760
2
votes
1
answer
280
views
Estimating transition probability from a state to itself with the MSM package
For a given panel dataset I have used the MSM package to estimate transition probabilities between states. Using the pnext.msm function I can obtain the ...
- 1,760
0
votes
1
answer
47
views
Transition Matrix for a non-trivial example
So I've just been introduced to Transition Matrices; and I was wondering what one for look like the following example:
"Symmetric random walk on the integers"
$S$ $=$ $\mathbb{Z}$, $\forall$ ...
- 161
1
vote
1
answer
99
views
"Forcing" equal probabilities in the matrix exponential of a Markov intensity matrix
I have an upper-right triangular transition intensity matrix $Q$ for a 7-state Markov model (with states $X_1,X_2,...,X_7$), from which I numerically calculate the matrix exponential to derive a ...
- 211
0
votes
0
answers
54
views
How to generate a realization from a transition matrix?
Consider a Markov chain of 4 states described by the transition matrix,
$$
T_{ij} =
\begin{bmatrix}
0.40 & 0.56 & 0.03 & 0.01\\
0.45 & 0.51 & 0.04 & 0.00\\
0.25 & 0.25 &...
- 11
3
votes
3
answers
805
views
Probability of doing a specific Path in a Markov Chain
My problem is the following: I have this graph, representing a Markov Chain:
For example, if I am in state 1, the probability of going in state 2 or 4 is $\frac{1}{2}$. So I'm saying that the ...
- 205
0
votes
1
answer
80
views
How to derive transition matrix in this stochastic process?
I am new to stochastic processes and trying to solve a question related to finding a transition matrix of some experiment. The question is a
A sequence of experiments is performed, in each of which ...
- 3
0
votes
0
answers
16
views
Why doesn’t two ways of calculating stationary distribution result in the same answer?
I use
matpow=function(M,n){
ans=M
for(i in 1:(n-1)){
ans=ans%*%M
}
ans
}
to set the matpow function.
then I enter the transition matrix
...
- 45
3
votes
1
answer
1k
views
Number of states in HMM
I am testing a HMM model by generating data from a 3x3 transition matrix and 3x4 emission matrix and then trying to train a HMM model against this data with different initializations. When I plot the ...
- 33
1
vote
0
answers
90
views
How to maximize the steady state transition probability for a state in a Markov chain by altering that state's outgoing transition probabilities?
Let's say we have a transition matrix of
which can be solved to come up with steady state transition probabilities of
Alpha: 34.9%
Beta: 24.0%
Gamma: 16.9%
Delta: 24.2%
Now, imagine Alpha, Beta, ...
0
votes
0
answers
75
views
Simulate discrete state space CTMC from generator matrix
Consider a generator matrix $Q\in\mathbb{R}^{h\times h}$ for a discrete state space $\{1,...,h\}$. I want to determine the probability of a single transition of a stochastic process $X(t)$ with $X(0)=...
user290384
0
votes
1
answer
412
views
How to calculate the probability Matrix (Alpha) for Regular Markov chains
Pardon me for being a novice here. In the image attached, eq 3.1 represents the transition matrix (it's pretty clear). I am not able to comprehend the eq 3.2, alpha*P = alpha, as well as the further ...
0
votes
2
answers
376
views
$\pi_i P^n_{i, j} =$ long-run proportion of time the chain is in $i$ and will be in $j$ after $n$ transitions?
I am currently studying the textbook Introduction to Probability Models by Sheldon M. Ross. Chapter 4.4 Long-Run Proportions and Limiting Probabilities says the following:
Because $\pi_i$ is the long-...
- 1,364
0
votes
1
answer
812
views
generate a realization from a transition matrix
Consider a markov chain of 4 states $\{S_1, S_2, S_3, S_4\}$ described by the transition matrix
$$
A = \begin{bmatrix}
.25 & .20 & .25 & .30 \\
.20 & .30 & .25 & .30 \\
....
- 205
1
vote
1
answer
27
views
Calculating limit law for matrix
I my notes on Markov chains, I am presented with the following matrix:
$$\mathcal{P} = \begin{bmatrix} 0.97 & 0.03 & 0 & 0 \\ 0.008 & 0.982 & 0.01 & 0 \\ 0.02 & 0 & 0....
- 1,364
1
vote
1
answer
70
views
Example where unique stationary law, which is an occupation law, but no limit law exists
I am currently learning about the balance equations, mass equation, limit law, occupation law and stationary law in Markov models. The following example is presented:
Example 2:
$$\mathcal{P} = \...
- 1,364
1
vote
1
answer
184
views
Computation of balance equation example in Markov model
I am studying some examples of balance equations for Markov models. I am presented with the following example:
$$\mathcal{P} = \begin{bmatrix} 0.2 & 0.3 & 0.5 \\ 0.1 & 0 & 0.9 \\ 0.55 ...
- 1,364
1
vote
1
answer
211
views
Markov models and occupation time
I'm presented with the following explanation and proof:
Let $(X_n)$ be a Markov chain, and fix a state $j \in S$.
Define indicator variables: For $n = 0, 1, \dots$, let
$$I_n(j) = \begin{cases} 1 &...
- 1,364
1
vote
1
answer
1k
views
Calculating 7-step transition matrix for example
In my notes on Markov processes, I am presented with two related examples:
Example 1:
Classify daily weather for some region as Sunny (state $1$), Cloudy (state $2$), or rainy (state $3$).
...
- 1,364
0
votes
1
answer
771
views
Proof that the $n$-step transition matrix is the $n$th power of $\mathcal{P}$
I am presented with the following theorem in the context of Markov chains and stochastic systems:
The $n$-step transition matrix is the $n$th power of $\mathcal{P}$:
$$\mathcal{P}^{(n)} = P^n.$$
And ...
- 1,364
1
vote
0
answers
33
views
Transition probability matrix
A gambler tosses a coin and a tetrahedron at each stage. If $H$, he receives the amount appearing at the face of the tetrahedron. If $T$, he pays the amount. The tetrahedron is fair, but probability ...
- 21
0
votes
0
answers
22
views
Find transition probability matrix
A box contains 3 balls. Each is either white or red. The game is to draw a ball from each period. If red is drawn, a white is replaced. But if white is drawn, all the balls in the box is replaced by ...
- 21
0
votes
1
answer
164
views
Number of stationary distributions of a Markov chain
How do i determine the number of stationary distributions that a Markov chain has if it is not irreducible or regular.
The transition matrix is
...
- 47
2
votes
1
answer
754
views
Estimating model for transition probabilities of a Markov Chain
Suppose that I have a Markov chain with $S$ states evolving over time. I have $S^2\times T$ values of the transition matrix, where $T$ is the number of time periods. I also have $K$ matrices $X$ of $T\...
- 1,440
4
votes
1
answer
251
views
How to show that the transition probability is equal to $\overline p_{ij} = \frac{P_{ij}}{\sum_{k\neq i}p_{ik}}$
(No new answers needed) I would like to award @whuber for his good answer with my bounty!
Suppose that $(X_n)_{n≥0}$ is Markov$(λ, P)$ but
that we only observe the process when it moves to a new ...
user255658
1
vote
1
answer
1k
views
Rows and columns of the one-step transition probability matrix
I am currently studying the textbook Introduction to Modeling and Analysis of Stochastic Systems, Second Edition, by V. G. Kulkarni. In a section on discrete-time Markov chains, the author introduces ...
- 1,364
0
votes
0
answers
89
views
Singular state transition probability matrix in David Silver's UCL Lesson 2
I'm studying David Silver's second lesson on reinforcement learning: https://www.youtube.com/watch?v=lfHX2hHRMVQ&list=PLqYmG7hTraZDM-OYHWgPebj2MfCFzFObQ&index=2 and the state transition ...
1
vote
0
answers
23
views
Specifying a multi-state model with unobservable terminal state
Suppose there is a multi-state process with three states, listed below and labelled as terminal/non-terminal and observable/unobservable:
Initializing: non-terminal observable
Active: non-terminal ...
- 3,753
0
votes
0
answers
236
views
Make sense of plotting a transition matrix
I'm studying statistics and I'm trying to understand markov chain topic. I'm using the package "markovchain" in R to obtain the stationary distribution.
From this transition matrix $M$:
<...
- 1
2
votes
1
answer
1k
views
Measure the distance between two probability transition matrices
I have a probability transition matrix $P$ that contains some values very close to zero. I want to sparsify this matrix by taking the k largest values for each row and setting the others to zero. For ...
- 111
0
votes
1
answer
85
views
Long run proportion of transitions in a Markov chain
Let $S$ be a set of states for a Markov chain and let $S^C$ be the remaining states. Explain the identity $$\sum_{i\in S}\sum_{j\in S^C}\pi_iP_{ij}=\sum_{i\in S^C}\sum_{j\in S}\pi_iP_{ij}$$
I know ...
- 1
1
vote
1
answer
544
views
Irreducible (communicating) classes [closed]
The Markov chain $(Xn; n\geq)$ has state-space $S = (0, 1, 2, . . .)$, with
$p_{i,0} = \frac{1}{4}$ and $p_{i,i+1} = \frac{3}{4}$ $\forall i \geq 0$, so that the transition matrix is
P =$\...
1
vote
2
answers
648
views
Biased coins and Markov processes
Good day, I am attempting an optional exercise and I am finding it hard to interpret the problem in terms of matrices and vectors.
Coin 1 has probability 0.4 of coming up heads, and coin 2 has ...
0
votes
0
answers
116
views
Formulating a Transition matrix for Markov Process
I am dealing with a medical process which is as follows.
There are 10000 Veterans who are enrolled in this study.
All 10000 have medical condition called onychocryptosis which is a fancy term for ...
- 409