Questions tagged [transition-matrix]

A transition matrix is a square matrix used to describe the transitions of a Markov chain.

Filter by
Sorted by
Tagged with
2 votes
2 answers
172 views

How to understand the definition of Markov Chain $P(X_{n+1}\in B\mid \mathcal{F}_n)=p(X_n,B)$?

The definition of Markov Chain in Durrett (Probability: Theory and Examples, 2019, Section 5.2) is: $$P(X_{n+1}\in B\mid \mathcal{F}_n)=p(X_n,B), $$ where $p$ is the Markov transition kernel ...
dodo's user avatar
  • 185
0 votes
0 answers
39 views

Smoothing a transition matrix

I have a dataset containing observations at time t and t+1 of ratings A (best) to E (worst) and Default. I want to use transition matrices to predict future ratings t+2, t+3, etc The transition matrix ...
Lucy Edwards's user avatar
0 votes
0 answers
37 views

Transition Probabilities and Weibull Regression

I am currently reading a book on health technology assessment: Decision Modelling for Health Economic Evaluation (Briggs et al). Given all information below, I'd appreciate if someone could show how ...
EB3112's user avatar
  • 244
0 votes
0 answers
13 views

Inhomogeneous Hidden Markov Model

I've recently been working on Inhomogeneous Hidden Markov Model (IHMM) as defined in this article. In short, the whole idea distinguishing IHMM from typical HMM is that a transition matrix $A = \{a_{...
thesecond's user avatar
  • 350
0 votes
0 answers
20 views

transition probabilities of a markov process

I have a stochastic volatility model for commodity price which follows an AR(1) process: ln(pt ) − m = ρ (ln(pt−1) − m) + exp(σt)ut ut ∼ IID(0, 1) σt − μ = ρσ(σt−1 − μ) + ηεt εt ∼ IID(0, 1) ...
Nusrat's user avatar
  • 1
0 votes
0 answers
31 views

How to calculate the MLE for transition probabilities of markov chains of higher order?

my question is: I need to calculate the MLE for the transition probabilities of a Markok chain of order 2 ≤ k < ∞. I've calculated the MLE for Markov chain of order 1 on a previous exercise; but I ...
Giane Higino's user avatar
7 votes
2 answers
766 views

statistical comparison of two markov chain transition matrices

I'm working on a time-dependent dataset, where basically I have two different populations and we're building Markov chains to describe their behaviors. What I'm trying to do is compare the transition ...
Greg Markowsky's user avatar
3 votes
2 answers
649 views

How could I estimate a transition probability matrix that varies over time?

I have multiple Markov chains with twelve states. I want to estimate a transition probability matrix for each time point (except for the last time point) that can vary over time using all Markov ...
Captain Ahab's user avatar
0 votes
0 answers
50 views

stationary distribution of a continuous time markov chain

With a certain rate $R$ balls fall into a box. There is no limit to the number of balls the box can hold, but each ball has a rate $\gamma$ to leave the box and when two balls hit each other they ...
bawo__'s user avatar
  • 1
1 vote
0 answers
32 views

In Markov process, how to solve $P(X_n=1)$

Let $X_n$ is Markov chain for $n\geq 0$ and state space is $E=\{1,2,3\}$. One-step transition probability matrix is $$ \left[ \begin{matrix} p_{11}& p_{12}& p_{13}\\ p_{21}& p_{22}&...
Zeta's user avatar
  • 11
1 vote
1 answer
176 views

Estimating Markov chain transition probabilities from data

In a discrete time and space Markov chain, I know the formula to estimate the transition probabilities $$p_{ij} = \frac{n_{ij}}{\sum_{j \in S} n_{ij}}$$ I'm not sure however how you can find this is ...
Jonathan's user avatar
1 vote
1 answer
95 views

Markov chain: inferring transition rates from equilibrium

I feel like I have been having a very dumb week trying to solve/research this problem and that I am missing an easy solution or that it is not possible. Given an equilibrium distribution (say from a ...
markovian's user avatar
1 vote
0 answers
167 views

How to prove a sequence of random variables dependent on different random variables is a homogeneous Markov chain?

In Pierre Brémaud's book, Markov Chains - Gibbs Fields, Monte Carlo Simulation and Queues, exercise 2.6.9 is stated as follows: Let $\{Z_n\}_{n \geq 1}$ be an IID sequence of geometric random ...
Jxson99's user avatar
  • 664
1 vote
1 answer
1k views

Coin flipping (Markov chain)

Suppose that coin 1 has probability 0.7 of coming up heads, and coin 2 has probability 0.6 of coming up heads. If the coin flipped today comes up heads, then we select coin 1 to flip tomorrow, and if ...
Win_odd Dhamnekar's user avatar
1 vote
0 answers
30 views

How to decide $X_n, n \in [0,\infty)$ is a Markov chain and how to compute transition probability matrix?

Three white and three black balls are distributed in two urns in such a way that each contains three balls. We say that the system is in state i, i = 0, 1, 2, 3, if the first urn contains i white ...
Win_odd Dhamnekar's user avatar
0 votes
0 answers
140 views

Is this transition probability matrix correct?

A pensioner receives 2 (thousand dollars) at the beginning of each month. The amount of money he needs to spend during a month is independent of the amount he has and is equal to i with probability $...
Win_odd Dhamnekar's user avatar
1 vote
0 answers
104 views

How to map a sequence to a transition matrix

I have the following transition matrices, one for Maria and one for Anna: ...
quant's user avatar
  • 511
0 votes
1 answer
287 views

What are the transition functions for RNNs

From what I understand, the hidden states of RNNs are equivalent to the deterministic probability distribution over hidden states in for example a Hidden Markov Model. Thus, just as probabilistic ...
math189925's user avatar
0 votes
0 answers
117 views

R CODE Transition Matrix having zeroes for some transitions

I have been trying to create a Transition Matrix using the data from 2000 entities over 40 observations (Years). I have ranked the data into percentiles, for example the highest value entity in a ...
Hermann Josef's user avatar
0 votes
0 answers
68 views

Estimating transition probabilities of a Markov chain with bayesian approach

I am trying to model heteroskedasticity in time series data,and the volatility $\left(\sigma_{t}\right)$ is taken as a Markov chain with two values $\sigma_{h}>\sigma_{l}>0$ with transition ...
Mrnobody's user avatar
1 vote
0 answers
100 views

Approximating a countable-state (infinite) Markov model with a finite-state one

TL;DR—in a nutshell I have a countable-state Markov model (with a countably infinite number of states) in which the probability of transitioning to states $S_{i>k}$ for large $k$s are practically ...
psyguy's user avatar
  • 311
1 vote
1 answer
251 views

R MSM: unexpected identical transition rate matrices for different values of covariate; why?

I am fitting a continuous-time Markov model to a panel dataset using the R package MSM. Because I am interested in sex-differences in transition rates, I fit the ...
Stijn's user avatar
  • 1,880
2 votes
1 answer
436 views

Estimating transition probability from a state to itself with the MSM package

For a given panel dataset I have used the MSM package to estimate transition probabilities between states. Using the pnext.msm function I can obtain the ...
Stijn's user avatar
  • 1,880
1 vote
1 answer
49 views

Transition Matrix for a non-trivial example

So I've just been introduced to Transition Matrices; and I was wondering what one for look like the following example: "Symmetric random walk on the integers" $S$ $=$ $\mathbb{Z}$, $\forall$ ...
Sam Connell's user avatar
1 vote
1 answer
116 views

"Forcing" equal probabilities in the matrix exponential of a Markov intensity matrix

I have an upper-right triangular transition intensity matrix $Q$ for a 7-state Markov model (with states $X_1,X_2,...,X_7$), from which I numerically calculate the matrix exponential to derive a ...
Rich Pollock's user avatar
0 votes
0 answers
109 views

How to generate a realization from a transition matrix?

Consider a Markov chain of 4 states described by the transition matrix, $$ T_{ij} = \begin{bmatrix} 0.40 & 0.56 & 0.03 & 0.01\\ 0.45 & 0.51 & 0.04 & 0.00\\ 0.25 & 0.25 &...
Advaita's user avatar
  • 11
3 votes
3 answers
1k views

Probability of doing a specific Path in a Markov Chain

My problem is the following: I have this graph, representing a Markov Chain: For example, if I am in state 1, the probability of going in state 2 or 4 is $\frac{1}{2}$. So I'm saying that the ...
docdev's user avatar
  • 205
0 votes
1 answer
107 views

How to derive transition matrix in this stochastic process?

I am new to stochastic processes and trying to solve a question related to finding a transition matrix of some experiment. The question is a A sequence of experiments is performed, in each of which ...
Maria's user avatar
  • 3
0 votes
0 answers
18 views

Why doesn’t two ways of calculating stationary distribution result in the same answer?

I use matpow=function(M,n){ ans=M for(i in 1:(n-1)){ ans=ans%*%M } ans } to set the matpow function. then I enter the transition matrix ...
merry123's user avatar
3 votes
1 answer
2k views

Number of states in HMM

I am testing a HMM model by generating data from a 3x3 transition matrix and 3x4 emission matrix and then trying to train a HMM model against this data with different initializations. When I plot the ...
lajisam's user avatar
  • 33
1 vote
0 answers
122 views

How to maximize the steady state transition probability for a state in a Markov chain by altering that state's outgoing transition probabilities?

Let's say we have a transition matrix of which can be solved to come up with steady state transition probabilities of Alpha: 34.9% Beta: 24.0% Gamma: 16.9% Delta: 24.2% Now, imagine Alpha, Beta, ...
Rob Goldstone's user avatar
0 votes
0 answers
92 views

Simulate discrete state space CTMC from generator matrix

Consider a generator matrix $Q\in\mathbb{R}^{h\times h}$ for a discrete state space $\{1,...,h\}$. I want to determine the probability of a single transition of a stochastic process $X(t)$ with $X(0)=...
user avatar
-1 votes
1 answer
534 views

How to calculate the probability Matrix (Alpha) for Regular Markov chains

Pardon me for being a novice here. In the image attached, eq 3.1 represents the transition matrix (it's pretty clear). I am not able to comprehend the eq 3.2, alpha*P = alpha, as well as the further ...
Mritunjay Kumar's user avatar
0 votes
2 answers
570 views

$\pi_i P^n_{i, j} =$ long-run proportion of time the chain is in $i$ and will be in $j$ after $n$ transitions?

I am currently studying the textbook Introduction to Probability Models by Sheldon M. Ross. Chapter 4.4 Long-Run Proportions and Limiting Probabilities says the following: Because $\pi_i$ is the long-...
The Pointer's user avatar
  • 1,752
0 votes
1 answer
1k views

generate a realization from a transition matrix

Consider a markov chain of 4 states $\{S_1, S_2, S_3, S_4\}$ described by the transition matrix $$ A = \begin{bmatrix} .25 & .20 & .25 & .30 \\ .20 & .30 & .25 & .30 \\ ....
mikeck's user avatar
  • 205
1 vote
1 answer
29 views

Calculating limit law for matrix

I my notes on Markov chains, I am presented with the following matrix: $$\mathcal{P} = \begin{bmatrix} 0.97 & 0.03 & 0 & 0 \\ 0.008 & 0.982 & 0.01 & 0 \\ 0.02 & 0 & 0....
The Pointer's user avatar
  • 1,752
1 vote
1 answer
78 views

Example where unique stationary law, which is an occupation law, but no limit law exists

I am currently learning about the balance equations, mass equation, limit law, occupation law and stationary law in Markov models. The following example is presented: Example 2: $$\mathcal{P} = \...
The Pointer's user avatar
  • 1,752
1 vote
1 answer
379 views

Computation of balance equation example in Markov model

I am studying some examples of balance equations for Markov models. I am presented with the following example: $$\mathcal{P} = \begin{bmatrix} 0.2 & 0.3 & 0.5 \\ 0.1 & 0 & 0.9 \\ 0.55 ...
The Pointer's user avatar
  • 1,752
1 vote
1 answer
376 views

Markov models and occupation time

I'm presented with the following explanation and proof: Let $(X_n)$ be a Markov chain, and fix a state $j \in S$. Define indicator variables: For $n = 0, 1, \dots$, let $$I_n(j) = \begin{cases} 1 &...
The Pointer's user avatar
  • 1,752
1 vote
1 answer
2k views

Calculating 7-step transition matrix for example

In my notes on Markov processes, I am presented with two related examples: Example 1: Classify daily weather for some region as Sunny (state $1$), Cloudy (state $2$), or rainy (state $3$). ...
The Pointer's user avatar
  • 1,752
0 votes
1 answer
1k views

Proof that the $n$-step transition matrix is the $n$th power of $\mathcal{P}$

I am presented with the following theorem in the context of Markov chains and stochastic systems: The $n$-step transition matrix is the $n$th power of $\mathcal{P}$: $$\mathcal{P}^{(n)} = P^n.$$ And ...
The Pointer's user avatar
  • 1,752
1 vote
0 answers
35 views

Transition probability matrix

A gambler tosses a coin and a tetrahedron at each stage. If $H$, he receives the amount appearing at the face of the tetrahedron. If $T$, he pays the amount. The tetrahedron is fair, but probability ...
user530299's user avatar
0 votes
0 answers
27 views

Find transition probability matrix

A box contains 3 balls. Each is either white or red. The game is to draw a ball from each period. If red is drawn, a white is replaced. But if white is drawn, all the balls in the box is replaced by ...
user530299's user avatar
0 votes
1 answer
215 views

Number of stationary distributions of a Markov chain

How do i determine the number of stationary distributions that a Markov chain has if it is not irreducible or regular. The transition matrix is ...
caston1414's user avatar
3 votes
1 answer
980 views

Estimating model for transition probabilities of a Markov Chain

Suppose that I have a Markov chain with $S$ states evolving over time. I have $S^2\times T$ values of the transition matrix, where $T$ is the number of time periods. I also have $K$ matrices $X$ of $T\...
DanielTheRocketMan's user avatar
4 votes
1 answer
304 views

How to show that the transition probability is equal to $\overline p_{ij} = \frac{P_{ij}}{\sum_{k\neq i}p_{ik}}$

(No new answers needed) I would like to award @whuber for his good answer with my bounty! Suppose that $(X_n)_{n≥0}$ is Markov$(λ, P)$ but that we only observe the process when it moves to a new ...
user avatar
1 vote
1 answer
2k views

Rows and columns of the one-step transition probability matrix

I am currently studying the textbook Introduction to Modeling and Analysis of Stochastic Systems, Second Edition, by V. G. Kulkarni. In a section on discrete-time Markov chains, the author introduces ...
The Pointer's user avatar
  • 1,752
0 votes
0 answers
129 views

Singular state transition probability matrix in David Silver's UCL Lesson 2

I'm studying David Silver's second lesson on reinforcement learning: https://www.youtube.com/watch?v=lfHX2hHRMVQ&list=PLqYmG7hTraZDM-OYHWgPebj2MfCFzFObQ&index=2 and the state transition ...
Ricardo Alberto's user avatar
1 vote
0 answers
26 views

Specifying a multi-state model with unobservable terminal state

Suppose there is a multi-state process with three states, listed below and labelled as terminal/non-terminal and observable/unobservable: Initializing: non-terminal observable Active: non-terminal ...
Brash Equilibrium's user avatar
0 votes
0 answers
254 views

Make sense of plotting a transition matrix

I'm studying statistics and I'm trying to understand markov chain topic. I'm using the package "markovchain" in R to obtain the stationary distribution. From this transition matrix $M$: <...
Paolo's user avatar
  • 1