Questions tagged [transition-matrix]
A transition matrix is a square matrix used to describe the transitions of a Markov chain.
108 questions
0
votes
0
answers
20
views
Estimating markov transition matrix using total elevator "ups" and "downs" by floor
I have data on elevator presses and I am hoping to use them to estimate a Markov transition matrix, so I can ultimately estimate how frequently people go to different floors.
For each floor from 1-4, ...
- 156
0
votes
0
answers
18
views
Is it possible to use Particle Marginal Metropolis Hastings to estimate the transition matrix and input?
A state space model is defined as:
$$x_{t+1} = A_tx_t + B_tu_t$$
$$y_{t+1} = H_tx_{t+1}$$
So my question is: is it possible to use Particle Marginal Metropolis Hastings to estimate the transition ...
- 75
0
votes
0
answers
71
views
Sum of powers (geometric series) of state transition matrix
I am working discrete time Markov chain analysis for some large state transition graph. I want to find the rewards/cost to reach from the init state to the terminal/accepting states.
I have the state ...
- 101
0
votes
0
answers
10
views
Which variable is best suited for edge weights when computing graph algorithms instead of relative risks?
I am currently trying to develop graph data. Which variable is best suited for edge weights when computing graph algorithms?
Relative risk
Relative Risk: Many networks in my field use relative risks ...
- 61
0
votes
1
answer
175
views
msm package: Mutlti state model initial value in 'vmmin' is not finite
I am new to msm package and markov models. I have a randomized trial dataset with readings from three time points: baseline, at 1 year, and at 2 year. I am trying to calculate annual transition ...
- 1
2
votes
2
answers
206
views
How to understand the definition of Markov Chain $P(X_{n+1}\in B\mid \mathcal{F}_n)=p(X_n,B)$?
The definition of Markov Chain in Durrett (Probability: Theory and Examples, 2019, Section 5.2) is:
$$P(X_{n+1}\in B\mid \mathcal{F}_n)=p(X_n,B), $$
where $p$ is the Markov transition kernel ...
- 185
0
votes
0
answers
117
views
Smoothing a transition matrix
I have a dataset containing observations at time t and t+1 of ratings A (best) to E (worst) and Default. I want to use transition matrices to predict future ratings t+2, t+3, etc
The transition matrix ...
8
votes
2
answers
1k
views
statistical comparison of two markov chain transition matrices
I'm working on a time-dependent dataset, where basically I have two different populations and we're building Markov chains to describe their behaviors. What I'm trying to do is compare the transition ...
3
votes
2
answers
1k
views
How could I estimate a transition probability matrix that varies over time?
I have multiple Markov chains with twelve states. I want to estimate a transition probability matrix for each time point (except for the last time point) that can vary over time using all Markov ...
- 65
1
vote
0
answers
37
views
In Markov process, how to solve $P(X_n=1)$
Let $X_n$ is Markov chain for $n\geq 0$ and state space is $E=\{1,2,3\}$. One-step transition probability matrix is
$$
\left[ \begin{matrix}
p_{11}& p_{12}& p_{13}\\
p_{21}& p_{22}&...
- 11
1
vote
1
answer
283
views
Estimating Markov chain transition probabilities from data
In a discrete time and space Markov chain, I know the formula to estimate the transition probabilities $$p_{ij} = \frac{n_{ij}}{\sum_{j \in S} n_{ij}}$$ I'm not sure however how you can find this is ...
- 13
1
vote
1
answer
139
views
Markov chain: inferring transition rates from equilibrium
I feel like I have been having a very dumb week trying to solve/research this problem and that I am missing an easy solution or that it is not possible. Given an equilibrium distribution (say from a ...
- 11
1
vote
0
answers
199
views
How to prove a sequence of random variables dependent on different random variables is a homogeneous Markov chain?
In Pierre Brémaud's book, Markov Chains - Gibbs Fields, Monte Carlo Simulation and Queues, exercise 2.6.9 is stated as follows:
Let $\{Z_n\}_{n \geq 1}$ be an IID sequence of geometric random ...
- 706
1
vote
1
answer
2k
views
Coin flipping (Markov chain)
Suppose that coin 1 has probability 0.7 of coming up heads, and coin 2 has probability 0.6 of coming up heads.
If the coin flipped today comes up heads, then we select coin 1 to flip tomorrow, and if ...
- 1,226
1
vote
0
answers
35
views
How to decide $X_n, n \in [0,\infty)$ is a Markov chain and how to compute transition probability matrix?
Three white and three black balls are distributed in two urns in such a way that each contains three balls. We say that the system is in state i, i = 0, 1, 2, 3, if the first urn contains i white ...
- 1,226
0
votes
0
answers
176
views
Is this transition probability matrix correct?
A pensioner receives 2 (thousand dollars) at the beginning of each month. The amount of money he needs to spend during a month is independent of the amount he has and is equal to i with probability $...
- 1,226
1
vote
0
answers
113
views
How to map a sequence to a transition matrix
I have the following transition matrices, one for Maria and one for Anna:
...
- 531
0
votes
1
answer
350
views
What are the transition functions for RNNs
From what I understand, the hidden states of RNNs are equivalent to the deterministic probability distribution over hidden states in for example a Hidden Markov Model.
Thus, just as probabilistic ...
- 135
0
votes
0
answers
124
views
R CODE Transition Matrix having zeroes for some transitions
I have been trying to create a Transition Matrix using the data from 2000 entities over 40 observations (Years). I have ranked the data into percentiles, for example the highest value entity in a ...
0
votes
0
answers
72
views
Estimating transition probabilities of a Markov chain with bayesian approach
I am trying to model heteroskedasticity in time series data,and the volatility $\left(\sigma_{t}\right)$ is taken as a Markov chain with two values $\sigma_{h}>\sigma_{l}>0$ with transition ...
- 3
1
vote
0
answers
104
views
Approximating a countable-state (infinite) Markov model with a finite-state one
TL;DR—in a nutshell
I have a countable-state Markov model (with a countably infinite number of states) in which the probability of transitioning to states $S_{i>k}$ for large $k$s are practically ...
- 311
1
vote
1
answer
293
views
R MSM: unexpected identical transition rate matrices for different values of covariate; why?
I am fitting a continuous-time Markov model to a panel dataset using the R package MSM. Because I am interested in sex-differences in transition rates, I fit the ...
- 1,922
2
votes
1
answer
520
views
Estimating transition probability from a state to itself with the MSM package
For a given panel dataset I have used the MSM package to estimate transition probabilities between states. Using the pnext.msm function I can obtain the ...
- 1,922
2
votes
1
answer
58
views
Transition Matrix for a non-trivial example
So I've just been introduced to Transition Matrices; and I was wondering what one for look like the following example:
"Symmetric random walk on the integers"
$S$ $=$ $\mathbb{Z}$, $\forall$ ...
- 381
1
vote
1
answer
130
views
"Forcing" equal probabilities in the matrix exponential of a Markov intensity matrix
I have an upper-right triangular transition intensity matrix $Q$ for a 7-state Markov model (with states $X_1,X_2,...,X_7$), from which I numerically calculate the matrix exponential to derive a ...
- 209
0
votes
0
answers
123
views
How to generate a realization from a transition matrix?
Consider a Markov chain of 4 states described by the transition matrix,
$$
T_{ij} =
\begin{bmatrix}
0.40 & 0.56 & 0.03 & 0.01\\
0.45 & 0.51 & 0.04 & 0.00\\
0.25 & 0.25 &...
- 11
3
votes
3
answers
1k
views
Probability of doing a specific Path in a Markov Chain
My problem is the following: I have this graph, representing a Markov Chain:
For example, if I am in state 1, the probability of going in state 2 or 4 is $\frac{1}{2}$. So I'm saying that the ...
- 205
0
votes
1
answer
116
views
How to derive transition matrix in this stochastic process?
I am new to stochastic processes and trying to solve a question related to finding a transition matrix of some experiment. The question is a
A sequence of experiments is performed, in each of which ...
- 3
0
votes
0
answers
19
views
Why doesn’t two ways of calculating stationary distribution result in the same answer?
I use
matpow=function(M,n){
ans=M
for(i in 1:(n-1)){
ans=ans%*%M
}
ans
}
to set the matpow function.
then I enter the transition matrix
...
- 45
3
votes
1
answer
2k
views
Number of states in HMM
I am testing a HMM model by generating data from a 3x3 transition matrix and 3x4 emission matrix and then trying to train a HMM model against this data with different initializations. When I plot the ...
- 33
1
vote
0
answers
129
views
How to maximize the steady state transition probability for a state in a Markov chain by altering that state's outgoing transition probabilities?
Let's say we have a transition matrix of
which can be solved to come up with steady state transition probabilities of
Alpha: 34.9%
Beta: 24.0%
Gamma: 16.9%
Delta: 24.2%
Now, imagine Alpha, Beta, ...
0
votes
0
answers
102
views
Simulate discrete state space CTMC from generator matrix
Consider a generator matrix $Q\in\mathbb{R}^{h\times h}$ for a discrete state space $\{1,...,h\}$. I want to determine the probability of a single transition of a stochastic process $X(t)$ with $X(0)=...
user290384
-1
votes
1
answer
552
views
How to calculate the probability Matrix (Alpha) for Regular Markov chains
Pardon me for being a novice here. In the image attached, eq 3.1 represents the transition matrix (it's pretty clear). I am not able to comprehend the eq 3.2, alpha*P = alpha, as well as the further ...
0
votes
2
answers
656
views
$\pi_i P^n_{i, j} =$ long-run proportion of time the chain is in $i$ and will be in $j$ after $n$ transitions?
I am currently studying the textbook Introduction to Probability Models by Sheldon M. Ross. Chapter 4.4 Long-Run Proportions and Limiting Probabilities says the following:
Because $\pi_i$ is the long-...
- 2,204
0
votes
1
answer
1k
views
generate a realization from a transition matrix
Consider a markov chain of 4 states $\{S_1, S_2, S_3, S_4\}$ described by the transition matrix
$$
A = \begin{bmatrix}
.25 & .20 & .25 & .30 \\
.20 & .30 & .25 & .30 \\
....
- 205
1
vote
1
answer
33
views
Calculating limit law for matrix
I my notes on Markov chains, I am presented with the following matrix:
$$\mathcal{P} = \begin{bmatrix} 0.97 & 0.03 & 0 & 0 \\ 0.008 & 0.982 & 0.01 & 0 \\ 0.02 & 0 & 0....
- 2,204
1
vote
1
answer
78
views
Example where unique stationary law, which is an occupation law, but no limit law exists
I am currently learning about the balance equations, mass equation, limit law, occupation law and stationary law in Markov models. The following example is presented:
Example 2:
$$\mathcal{P} = \...
- 2,204
1
vote
1
answer
439
views
Computation of balance equation example in Markov model
I am studying some examples of balance equations for Markov models. I am presented with the following example:
$$\mathcal{P} = \begin{bmatrix} 0.2 & 0.3 & 0.5 \\ 0.1 & 0 & 0.9 \\ 0.55 ...
- 2,204
1
vote
1
answer
518
views
Markov models and occupation time
I'm presented with the following explanation and proof:
Let $(X_n)$ be a Markov chain, and fix a state $j \in S$.
Define indicator variables: For $n = 0, 1, \dots$, let
$$I_n(j) = \begin{cases} 1 &...
- 2,204
1
vote
1
answer
2k
views
Calculating 7-step transition matrix for example
In my notes on Markov processes, I am presented with two related examples:
Example 1:
Classify daily weather for some region as Sunny (state $1$), Cloudy (state $2$), or rainy (state $3$).
...
- 2,204
0
votes
1
answer
1k
views
Proof that the $n$-step transition matrix is the $n$th power of $\mathcal{P}$
I am presented with the following theorem in the context of Markov chains and stochastic systems:
The $n$-step transition matrix is the $n$th power of $\mathcal{P}$:
$$\mathcal{P}^{(n)} = P^n.$$
And ...
- 2,204
1
vote
0
answers
35
views
Transition probability matrix
A gambler tosses a coin and a tetrahedron at each stage. If $H$, he receives the amount appearing at the face of the tetrahedron. If $T$, he pays the amount. The tetrahedron is fair, but probability ...
- 21
0
votes
0
answers
27
views
Find transition probability matrix
A box contains 3 balls. Each is either white or red. The game is to draw a ball from each period. If red is drawn, a white is replaced. But if white is drawn, all the balls in the box is replaced by ...
- 21
0
votes
1
answer
226
views
Number of stationary distributions of a Markov chain
How do i determine the number of stationary distributions that a Markov chain has if it is not irreducible or regular.
The transition matrix is
...
- 47
3
votes
1
answer
1k
views
Estimating model for transition probabilities of a Markov Chain
Suppose that I have a Markov chain with $S$ states evolving over time. I have $S^2\times T$ values of the transition matrix, where $T$ is the number of time periods. I also have $K$ matrices $X$ of $T\...
- 1,472
4
votes
1
answer
321
views
How to show that the transition probability is equal to $\overline p_{ij} = \frac{P_{ij}}{\sum_{k\neq i}p_{ik}}$
(No new answers needed) I would like to award @whuber for his good answer with my bounty!
Suppose that $(X_n)_{n≥0}$ is Markov$(λ, P)$ but
that we only observe the process when it moves to a new ...
user255658
1
vote
1
answer
2k
views
Rows and columns of the one-step transition probability matrix
I am currently studying the textbook Introduction to Modeling and Analysis of Stochastic Systems, Second Edition, by V. G. Kulkarni. In a section on discrete-time Markov chains, the author introduces ...
- 2,204
0
votes
0
answers
138
views
Singular state transition probability matrix in David Silver's UCL Lesson 2
I'm studying David Silver's second lesson on reinforcement learning: https://www.youtube.com/watch?v=lfHX2hHRMVQ&list=PLqYmG7hTraZDM-OYHWgPebj2MfCFzFObQ&index=2 and the state transition ...
1
vote
0
answers
30
views
Specifying a multi-state model with unobservable terminal state
Suppose there is a multi-state process with three states, listed below and labelled as terminal/non-terminal and observable/unobservable:
Initializing: non-terminal observable
Active: non-terminal ...
- 3,925
0
votes
0
answers
258
views
Make sense of plotting a transition matrix
I'm studying statistics and I'm trying to understand markov chain topic. I'm using the package "markovchain" in R to obtain the stationary distribution.
From this transition matrix $M$:
<...
- 1