Skip to main content

Questions tagged [transition-matrix]

A transition matrix is a square matrix used to describe the transitions of a Markov chain.

Filter by
Sorted by
Tagged with
0 votes
0 answers
19 views

Estimating markov transition matrix using total elevator "ups" and "downs" by floor

I have data on elevator presses and I am hoping to use them to estimate a Markov transition matrix, so I can ultimately estimate how frequently people go to different floors. For each floor from 1-4, ...
Jon Spring's user avatar
0 votes
0 answers
18 views

Is it possible to use Particle Marginal Metropolis Hastings to estimate the transition matrix and input?

A state space model is defined as: $$x_{t+1} = A_tx_t + B_tu_t$$ $$y_{t+1} = H_tx_{t+1}$$ So my question is: is it possible to use Particle Marginal Metropolis Hastings to estimate the transition ...
William Zhao's user avatar
0 votes
0 answers
50 views

Sum of powers (geometric series) of state transition matrix

I am working discrete time Markov chain analysis for some large state transition graph. I want to find the rewards/cost to reach from the init state to the terminal/accepting states. I have the state ...
JackDaniels's user avatar
0 votes
0 answers
4 views

Which variable is best suited for edge weights when computing graph algorithms instead of relative risks?

I am currently trying to develop graph data. Which variable is best suited for edge weights when computing graph algorithms? Relative risk Relative Risk: Many networks in my field use relative risks ...
user1190107's user avatar
0 votes
1 answer
84 views

msm package: Mutlti state model initial value in 'vmmin' is not finite

I am new to msm package and markov models. I have a randomized trial dataset with readings from three time points: baseline, at 1 year, and at 2 year. I am trying to calculate annual transition ...
spri0330's user avatar
2 votes
2 answers
195 views

How to understand the definition of Markov Chain $P(X_{n+1}\in B\mid \mathcal{F}_n)=p(X_n,B)$?

The definition of Markov Chain in Durrett (Probability: Theory and Examples, 2019, Section 5.2) is: $$P(X_{n+1}\in B\mid \mathcal{F}_n)=p(X_n,B), $$ where $p$ is the Markov transition kernel ...
dodo's user avatar
  • 185
0 votes
0 answers
84 views

Smoothing a transition matrix

I have a dataset containing observations at time t and t+1 of ratings A (best) to E (worst) and Default. I want to use transition matrices to predict future ratings t+2, t+3, etc The transition matrix ...
Lucy Edwards's user avatar
0 votes
0 answers
44 views

Transition Probabilities and Weibull Regression

I am currently reading a book on health technology assessment: Decision Modelling for Health Economic Evaluation (Briggs et al). Given all information below, I'd appreciate if someone could show how ...
EB3112's user avatar
  • 244
0 votes
0 answers
15 views

Inhomogeneous Hidden Markov Model

I've recently been working on Inhomogeneous Hidden Markov Model (IHMM) as defined in this article. In short, the whole idea distinguishing IHMM from typical HMM is that a transition matrix $A = \{a_{...
thesecond's user avatar
  • 390
7 votes
2 answers
1k views

statistical comparison of two markov chain transition matrices

I'm working on a time-dependent dataset, where basically I have two different populations and we're building Markov chains to describe their behaviors. What I'm trying to do is compare the transition ...
Greg Markowsky's user avatar
3 votes
2 answers
895 views

How could I estimate a transition probability matrix that varies over time?

I have multiple Markov chains with twelve states. I want to estimate a transition probability matrix for each time point (except for the last time point) that can vary over time using all Markov ...
Captain Ahab's user avatar
1 vote
0 answers
33 views

In Markov process, how to solve $P(X_n=1)$

Let $X_n$ is Markov chain for $n\geq 0$ and state space is $E=\{1,2,3\}$. One-step transition probability matrix is $$ \left[ \begin{matrix} p_{11}& p_{12}& p_{13}\\ p_{21}& p_{22}&...
Zeta's user avatar
  • 11
1 vote
1 answer
253 views

Estimating Markov chain transition probabilities from data

In a discrete time and space Markov chain, I know the formula to estimate the transition probabilities $$p_{ij} = \frac{n_{ij}}{\sum_{j \in S} n_{ij}}$$ I'm not sure however how you can find this is ...
Jonathan's user avatar
1 vote
1 answer
121 views

Markov chain: inferring transition rates from equilibrium

I feel like I have been having a very dumb week trying to solve/research this problem and that I am missing an easy solution or that it is not possible. Given an equilibrium distribution (say from a ...
markovian's user avatar
1 vote
0 answers
180 views

How to prove a sequence of random variables dependent on different random variables is a homogeneous Markov chain?

In Pierre Brémaud's book, Markov Chains - Gibbs Fields, Monte Carlo Simulation and Queues, exercise 2.6.9 is stated as follows: Let $\{Z_n\}_{n \geq 1}$ be an IID sequence of geometric random ...
Jxson99's user avatar
  • 664
1 vote
1 answer
2k views

Coin flipping (Markov chain)

Suppose that coin 1 has probability 0.7 of coming up heads, and coin 2 has probability 0.6 of coming up heads. If the coin flipped today comes up heads, then we select coin 1 to flip tomorrow, and if ...
Win_odd Dhamnekar's user avatar
1 vote
0 answers
32 views

How to decide $X_n, n \in [0,\infty)$ is a Markov chain and how to compute transition probability matrix?

Three white and three black balls are distributed in two urns in such a way that each contains three balls. We say that the system is in state i, i = 0, 1, 2, 3, if the first urn contains i white ...
Win_odd Dhamnekar's user avatar
0 votes
0 answers
156 views

Is this transition probability matrix correct?

A pensioner receives 2 (thousand dollars) at the beginning of each month. The amount of money he needs to spend during a month is independent of the amount he has and is equal to i with probability $...
Win_odd Dhamnekar's user avatar
1 vote
0 answers
108 views

How to map a sequence to a transition matrix

I have the following transition matrices, one for Maria and one for Anna: ...
quant's user avatar
  • 521
0 votes
1 answer
326 views

What are the transition functions for RNNs

From what I understand, the hidden states of RNNs are equivalent to the deterministic probability distribution over hidden states in for example a Hidden Markov Model. Thus, just as probabilistic ...
math189925's user avatar
0 votes
0 answers
118 views

R CODE Transition Matrix having zeroes for some transitions

I have been trying to create a Transition Matrix using the data from 2000 entities over 40 observations (Years). I have ranked the data into percentiles, for example the highest value entity in a ...
Hermann Josef's user avatar
0 votes
0 answers
69 views

Estimating transition probabilities of a Markov chain with bayesian approach

I am trying to model heteroskedasticity in time series data,and the volatility $\left(\sigma_{t}\right)$ is taken as a Markov chain with two values $\sigma_{h}>\sigma_{l}>0$ with transition ...
Mrnobody's user avatar
1 vote
0 answers
102 views

Approximating a countable-state (infinite) Markov model with a finite-state one

TL;DR—in a nutshell I have a countable-state Markov model (with a countably infinite number of states) in which the probability of transitioning to states $S_{i>k}$ for large $k$s are practically ...
psyguy's user avatar
  • 311
1 vote
1 answer
274 views

R MSM: unexpected identical transition rate matrices for different values of covariate; why?

I am fitting a continuous-time Markov model to a panel dataset using the R package MSM. Because I am interested in sex-differences in transition rates, I fit the ...
Stijn's user avatar
  • 1,892
2 votes
1 answer
493 views

Estimating transition probability from a state to itself with the MSM package

For a given panel dataset I have used the MSM package to estimate transition probabilities between states. Using the pnext.msm function I can obtain the ...
Stijn's user avatar
  • 1,892
2 votes
1 answer
53 views

Transition Matrix for a non-trivial example

So I've just been introduced to Transition Matrices; and I was wondering what one for look like the following example: "Symmetric random walk on the integers" $S$ $=$ $\mathbb{Z}$, $\forall$ ...
Sam Connell's user avatar
1 vote
1 answer
124 views

"Forcing" equal probabilities in the matrix exponential of a Markov intensity matrix

I have an upper-right triangular transition intensity matrix $Q$ for a 7-state Markov model (with states $X_1,X_2,...,X_7$), from which I numerically calculate the matrix exponential to derive a ...
Rich Pollock's user avatar
0 votes
0 answers
116 views

How to generate a realization from a transition matrix?

Consider a Markov chain of 4 states described by the transition matrix, $$ T_{ij} = \begin{bmatrix} 0.40 & 0.56 & 0.03 & 0.01\\ 0.45 & 0.51 & 0.04 & 0.00\\ 0.25 & 0.25 &...
Advaita's user avatar
  • 11
3 votes
3 answers
1k views

Probability of doing a specific Path in a Markov Chain

My problem is the following: I have this graph, representing a Markov Chain: For example, if I am in state 1, the probability of going in state 2 or 4 is $\frac{1}{2}$. So I'm saying that the ...
docdev's user avatar
  • 205
0 votes
1 answer
111 views

How to derive transition matrix in this stochastic process?

I am new to stochastic processes and trying to solve a question related to finding a transition matrix of some experiment. The question is a A sequence of experiments is performed, in each of which ...
Maria's user avatar
  • 3
0 votes
0 answers
18 views

Why doesn’t two ways of calculating stationary distribution result in the same answer?

I use matpow=function(M,n){ ans=M for(i in 1:(n-1)){ ans=ans%*%M } ans } to set the matpow function. then I enter the transition matrix ...
merry123's user avatar
3 votes
1 answer
2k views

Number of states in HMM

I am testing a HMM model by generating data from a 3x3 transition matrix and 3x4 emission matrix and then trying to train a HMM model against this data with different initializations. When I plot the ...
lajisam's user avatar
  • 33
1 vote
0 answers
125 views

How to maximize the steady state transition probability for a state in a Markov chain by altering that state's outgoing transition probabilities?

Let's say we have a transition matrix of which can be solved to come up with steady state transition probabilities of Alpha: 34.9% Beta: 24.0% Gamma: 16.9% Delta: 24.2% Now, imagine Alpha, Beta, ...
Rob Goldstone's user avatar
0 votes
0 answers
97 views

Simulate discrete state space CTMC from generator matrix

Consider a generator matrix $Q\in\mathbb{R}^{h\times h}$ for a discrete state space $\{1,...,h\}$. I want to determine the probability of a single transition of a stochastic process $X(t)$ with $X(0)=...
user avatar
-1 votes
1 answer
543 views

How to calculate the probability Matrix (Alpha) for Regular Markov chains

Pardon me for being a novice here. In the image attached, eq 3.1 represents the transition matrix (it's pretty clear). I am not able to comprehend the eq 3.2, alpha*P = alpha, as well as the further ...
Mritunjay Kumar's user avatar
0 votes
2 answers
615 views

$\pi_i P^n_{i, j} =$ long-run proportion of time the chain is in $i$ and will be in $j$ after $n$ transitions?

I am currently studying the textbook Introduction to Probability Models by Sheldon M. Ross. Chapter 4.4 Long-Run Proportions and Limiting Probabilities says the following: Because $\pi_i$ is the long-...
The Pointer's user avatar
  • 2,096
0 votes
1 answer
1k views

generate a realization from a transition matrix

Consider a markov chain of 4 states $\{S_1, S_2, S_3, S_4\}$ described by the transition matrix $$ A = \begin{bmatrix} .25 & .20 & .25 & .30 \\ .20 & .30 & .25 & .30 \\ ....
mikeck's user avatar
  • 205
1 vote
1 answer
31 views

Calculating limit law for matrix

I my notes on Markov chains, I am presented with the following matrix: $$\mathcal{P} = \begin{bmatrix} 0.97 & 0.03 & 0 & 0 \\ 0.008 & 0.982 & 0.01 & 0 \\ 0.02 & 0 & 0....
The Pointer's user avatar
  • 2,096
1 vote
1 answer
78 views

Example where unique stationary law, which is an occupation law, but no limit law exists

I am currently learning about the balance equations, mass equation, limit law, occupation law and stationary law in Markov models. The following example is presented: Example 2: $$\mathcal{P} = \...
The Pointer's user avatar
  • 2,096
1 vote
1 answer
409 views

Computation of balance equation example in Markov model

I am studying some examples of balance equations for Markov models. I am presented with the following example: $$\mathcal{P} = \begin{bmatrix} 0.2 & 0.3 & 0.5 \\ 0.1 & 0 & 0.9 \\ 0.55 ...
The Pointer's user avatar
  • 2,096
1 vote
1 answer
459 views

Markov models and occupation time

I'm presented with the following explanation and proof: Let $(X_n)$ be a Markov chain, and fix a state $j \in S$. Define indicator variables: For $n = 0, 1, \dots$, let $$I_n(j) = \begin{cases} 1 &...
The Pointer's user avatar
  • 2,096
1 vote
1 answer
2k views

Calculating 7-step transition matrix for example

In my notes on Markov processes, I am presented with two related examples: Example 1: Classify daily weather for some region as Sunny (state $1$), Cloudy (state $2$), or rainy (state $3$). ...
The Pointer's user avatar
  • 2,096
0 votes
1 answer
1k views

Proof that the $n$-step transition matrix is the $n$th power of $\mathcal{P}$

I am presented with the following theorem in the context of Markov chains and stochastic systems: The $n$-step transition matrix is the $n$th power of $\mathcal{P}$: $$\mathcal{P}^{(n)} = P^n.$$ And ...
The Pointer's user avatar
  • 2,096
1 vote
0 answers
35 views

Transition probability matrix

A gambler tosses a coin and a tetrahedron at each stage. If $H$, he receives the amount appearing at the face of the tetrahedron. If $T$, he pays the amount. The tetrahedron is fair, but probability ...
user530299's user avatar
0 votes
0 answers
27 views

Find transition probability matrix

A box contains 3 balls. Each is either white or red. The game is to draw a ball from each period. If red is drawn, a white is replaced. But if white is drawn, all the balls in the box is replaced by ...
user530299's user avatar
0 votes
1 answer
223 views

Number of stationary distributions of a Markov chain

How do i determine the number of stationary distributions that a Markov chain has if it is not irreducible or regular. The transition matrix is ...
caston1414's user avatar
3 votes
1 answer
1k views

Estimating model for transition probabilities of a Markov Chain

Suppose that I have a Markov chain with $S$ states evolving over time. I have $S^2\times T$ values of the transition matrix, where $T$ is the number of time periods. I also have $K$ matrices $X$ of $T\...
DanielTheRocketMan's user avatar
4 votes
1 answer
315 views

How to show that the transition probability is equal to $\overline p_{ij} = \frac{P_{ij}}{\sum_{k\neq i}p_{ik}}$

(No new answers needed) I would like to award @whuber for his good answer with my bounty! Suppose that $(X_n)_{n≥0}$ is Markov$(λ, P)$ but that we only observe the process when it moves to a new ...
user avatar
1 vote
1 answer
2k views

Rows and columns of the one-step transition probability matrix

I am currently studying the textbook Introduction to Modeling and Analysis of Stochastic Systems, Second Edition, by V. G. Kulkarni. In a section on discrete-time Markov chains, the author introduces ...
The Pointer's user avatar
  • 2,096
0 votes
0 answers
135 views

Singular state transition probability matrix in David Silver's UCL Lesson 2

I'm studying David Silver's second lesson on reinforcement learning: https://www.youtube.com/watch?v=lfHX2hHRMVQ&list=PLqYmG7hTraZDM-OYHWgPebj2MfCFzFObQ&index=2 and the state transition ...
Ricardo Alberto's user avatar