Questions tagged [transition-matrix]

A transition matrix is a square matrix used to describe the transitions of a Markov chain.

Filter by
Sorted by
Tagged with
1 vote
0 answers
11 views

How to map a sequence to a transition matrix

I have the following transition matrices, one for Maria and one for Anna: ...
user avatar
  • 359
0 votes
0 answers
12 views

How to analyze a time series of categorical data?

I have some data that look like this: ...
user avatar
  • 359
0 votes
1 answer
37 views

What are the transition functions for RNNs

From what I understand, the hidden states of RNNs are equivalent to the deterministic probability distribution over hidden states in for example a Hidden Markov Model. Thus, just as probabilistic ...
user avatar
0 votes
0 answers
51 views

R CODE Transition Matrix having zeroes for some transitions

I have been trying to create a Transition Matrix using the data from 2000 entities over 40 observations (Years). I have ranked the data into percentiles, for example the highest value entity in a ...
user avatar
0 votes
0 answers
23 views

Estimating transition probabilities of a Markov chain with bayesian approach

I am trying to model heteroskedasticity in time series data,and the volatility $\left(\sigma_{t}\right)$ is taken as a Markov chain with two values $\sigma_{h}>\sigma_{l}>0$ with transition ...
user avatar
1 vote
0 answers
39 views

Approximating a countable-state (infinite) Markov model with a finite-state one

TL;DR—in a nutshell I have a countable-state Markov model (with a countably infinite number of states) in which the probability of transitioning to states $S_{i>k}$ for large $k$s are practically ...
user avatar
  • 159
0 votes
1 answer
38 views

R MSM: unexpected identical transition rate matrices for different values of covariate; why?

I am fitting a continuous-time Markov model to a panel dataset using the R package MSM. Because I am interested in sex-differences in transition rates, I fit the ...
user avatar
  • 1,560
0 votes
0 answers
24 views

What is the relationship between 2 covariance matrices of 2 variables?

Assume variable $\textbf{x} = [x_1, x_2]^\top$ with mean $\mathbf{\mu} = [\mu_1, \mu_2]^\top$ and covariance matrix: \begin{equation} \Sigma_{\textbf{x}} = \begin{bmatrix} \sigma_1^2 & \text{cov}(\...
user avatar
1 vote
1 answer
96 views

Estimating transition probability from a state to itself with the MSM package

For a given panel dataset I have used the MSM package to estimate transition probabilities between states. Using the pnext.msm function I can obtain the ...
user avatar
  • 1,560
0 votes
1 answer
42 views

Transition Matrix for a non-trivial example

So I've just been introduced to Transition Matrices; and I was wondering what one for look like the following example: "Symmetric random walk on the integers" $S$ $=$ $\mathbb{Z}$, $\forall$ ...
user avatar
1 vote
1 answer
60 views

"Forcing" equal probabilities in the matrix exponential of a Markov intensity matrix

I have an upper-right triangular transition intensity matrix $Q$ for a 7-state Markov model (with states $X_1,X_2,...,X_7$), from which I numerically calculate the matrix exponential to derive a ...
user avatar
0 votes
0 answers
35 views

How to generate a realization from a transition matrix?

Consider a Markov chain of 4 states described by the transition matrix, $$ T_{ij} = \begin{bmatrix} 0.40 & 0.56 & 0.03 & 0.01\\ 0.45 & 0.51 & 0.04 & 0.00\\ 0.25 & 0.25 &...
user avatar
  • 11
2 votes
3 answers
488 views

Probability of doing a specific Path in a Markov Chain

My problem is the following: I have this graph, representing a Markov Chain: For example, if I am in state 1, the probability of going in state 2 or 4 is $\frac{1}{2}$. So I'm saying that the ...
user avatar
  • 195
0 votes
1 answer
35 views

How to derive transition matrix in this stochastic process?

I am new to stochastic processes and trying to solve a question related to finding a transition matrix of some experiment. The question is a A sequence of experiments is performed, in each of which ...
user avatar
  • 3
0 votes
0 answers
11 views

Why doesn’t two ways of calculating stationary distribution result in the same answer?

I use matpow=function(M,n){ ans=M for(i in 1:(n-1)){ ans=ans%*%M } ans } to set the matpow function. then I enter the transition matrix ...
user avatar
2 votes
1 answer
615 views

Number of states in HMM

I am testing a HMM model by generating data from a 3x3 transition matrix and 3x4 emission matrix and then trying to train a HMM model against this data with different initializations. When I plot the ...
user avatar
  • 23
1 vote
0 answers
48 views

How to maximize the steady state transition probability for a state in a Markov chain by altering that state's outgoing transition probabilities?

Let's say we have a transition matrix of which can be solved to come up with steady state transition probabilities of Alpha: 34.9% Beta: 24.0% Gamma: 16.9% Delta: 24.2% Now, imagine Alpha, Beta, ...
user avatar
0 votes
0 answers
38 views

Simulate discrete state space CTMC from generator matrix

Consider a generator matrix $Q\in\mathbb{R}^{h\times h}$ for a discrete state space $\{1,...,h\}$. I want to determine the probability of a single transition of a stochastic process $X(t)$ with $X(0)=...
user avatar
0 votes
2 answers
194 views

How to calculate the probability Matrix (Alpha) for Regular Markov chains

Pardon me for being a novice here. In the image attached, eq 3.1 represents the transition matrix (it's pretty clear). I am not able to comprehend the eq 3.2, alpha*P = alpha, as well as the further ...
user avatar
0 votes
2 answers
264 views

$\pi_i P^n_{i, j} =$ long-run proportion of time the chain is in $i$ and will be in $j$ after $n$ transitions?

I am currently studying the textbook Introduction to Probability Models by Sheldon M. Ross. Chapter 4.4 Long-Run Proportions and Limiting Probabilities says the following: Because $\pi_i$ is the long-...
user avatar
  • 1,096
0 votes
1 answer
413 views

generate a realization from a transition matrix

Consider a markov chain of 4 states $\{S_1, S_2, S_3, S_4\}$ described by the transition matrix $$ A = \begin{bmatrix} .25 & .20 & .25 & .30 \\ .20 & .30 & .25 & .30 \\ ....
user avatar
  • 205
1 vote
1 answer
27 views

Calculating limit law for matrix

I my notes on Markov chains, I am presented with the following matrix: $$\mathcal{P} = \begin{bmatrix} 0.97 & 0.03 & 0 & 0 \\ 0.008 & 0.982 & 0.01 & 0 \\ 0.02 & 0 & 0....
user avatar
  • 1,096
1 vote
1 answer
64 views

Example where unique stationary law, which is an occupation law, but no limit law exists

I am currently learning about the balance equations, mass equation, limit law, occupation law and stationary law in Markov models. The following example is presented: Example 2: $$\mathcal{P} = \...
user avatar
  • 1,096
1 vote
1 answer
48 views

Computation of balance equation example in Markov model

I am studying some examples of balance equations for Markov models. I am presented with the following example: $$\mathcal{P} = \begin{bmatrix} 0.2 & 0.3 & 0.5 \\ 0.1 & 0 & 0.9 \\ 0.55 ...
user avatar
  • 1,096
1 vote
1 answer
123 views

Markov models and occupation time

I'm presented with the following explanation and proof: Let $(X_n)$ be a Markov chain, and fix a state $j \in S$. Define indicator variables: For $n = 0, 1, \dots$, let $$I_n(j) = \begin{cases} 1 &...
user avatar
  • 1,096
1 vote
1 answer
522 views

Calculating 7-step transition matrix for example

In my notes on Markov processes, I am presented with two related examples: Example 1: Classify daily weather for some region as Sunny (state $1$), Cloudy (state $2$), or rainy (state $3$). ...
user avatar
  • 1,096
0 votes
1 answer
443 views

Proof that the $n$-step transition matrix is the $n$th power of $\mathcal{P}$

I am presented with the following theorem in the context of Markov chains and stochastic systems: The $n$-step transition matrix is the $n$th power of $\mathcal{P}$: $$\mathcal{P}^{(n)} = P^n.$$ And ...
user avatar
  • 1,096
1 vote
0 answers
31 views

Transition probability matrix

A gambler tosses a coin and a tetrahedron at each stage. If $H$, he receives the amount appearing at the face of the tetrahedron. If $T$, he pays the amount. The tetrahedron is fair, but probability ...
user avatar
0 votes
0 answers
17 views

Find transition probability matrix

A box contains 3 balls. Each is either white or red. The game is to draw a ball from each period. If red is drawn, a white is replaced. But if white is drawn, all the balls in the box is replaced by ...
user avatar
0 votes
1 answer
109 views

Number of stationary distributions of a Markov chain

How do i determine the number of stationary distributions that a Markov chain has if it is not irreducible or regular. The transition matrix is ...
user avatar
2 votes
1 answer
455 views

Estimating model for transition probabilities of a Markov Chain

Suppose that I have a Markov chain with $S$ states evolving over time. I have $S^2\times T$ values of the transition matrix, where $T$ is the number of time periods. I also have $K$ matrices $X$ of $T\...
user avatar
4 votes
1 answer
192 views

How to show that the transition probability is equal to $\overline p_{ij} = \frac{P_{ij}}{\sum_{k\neq i}p_{ik}}$

(No new answers needed) I would like to award @whuber for his good answer with my bounty! Suppose that $(X_n)_{n≥0}$ is Markov$(λ, P)$ but that we only observe the process when it moves to a new ...
user avatar
1 vote
1 answer
575 views

Rows and columns of the one-step transition probability matrix

I am currently studying the textbook Introduction to Modeling and Analysis of Stochastic Systems, Second Edition, by V. G. Kulkarni. In a section on discrete-time Markov chains, the author introduces ...
user avatar
  • 1,096
0 votes
0 answers
51 views

Singular state transition probability matrix in David Silver's UCL Lesson 2

I'm studying David Silver's second lesson on reinforcement learning: https://www.youtube.com/watch?v=lfHX2hHRMVQ&list=PLqYmG7hTraZDM-OYHWgPebj2MfCFzFObQ&index=2 and the state transition ...
user avatar
1 vote
0 answers
21 views

Specifying a multi-state model with unobservable terminal state

Suppose there is a multi-state process with three states, listed below and labelled as terminal/non-terminal and observable/unobservable: Initializing: non-terminal observable Active: non-terminal ...
user avatar
0 votes
0 answers
201 views

Make sense of plotting a transition matrix

I'm studying statistics and I'm trying to understand markov chain topic. I'm using the package "markovchain" in R to obtain the stationary distribution. From this transition matrix $M$: <...
user avatar
  • 1
2 votes
1 answer
739 views

Measure the distance between two probability transition matrices

I have a probability transition matrix $P$ that contains some values very close to zero. I want to sparsify this matrix by taking the k largest values for each row and setting the others to zero. For ...
user avatar
0 votes
1 answer
71 views

Long run proportion of transitions in a Markov chain

Let $S$ be a set of states for a Markov chain and let $S^C$ be the remaining states. Explain the identity $$\sum_{i\in S}\sum_{j\in S^C}\pi_iP_{ij}=\sum_{i\in S^C}\sum_{j\in S}\pi_iP_{ij}$$ I know ...
user avatar
1 vote
1 answer
473 views

Irreducible (communicating) classes [closed]

The Markov chain $(Xn; n\geq)$ has state-space $S = (0, 1, 2, . . .)$, with $p_{i,0} = \frac{1}{4}$ and $p_{i,i+1} = \frac{3}{4}$ $\forall i \geq 0$, so that the transition matrix is P =$\...
user avatar
1 vote
2 answers
497 views

Biased coins and Markov processes

Good day, I am attempting an optional exercise and I am finding it hard to interpret the problem in terms of matrices and vectors. Coin 1 has probability 0.4 of coming up heads, and coin 2 has ...
user avatar
0 votes
0 answers
106 views

Formulating a Transition matrix for Markov Process

I am dealing with a medical process which is as follows. There are 10000 Veterans who are enrolled in this study. All 10000 have medical condition called onychocryptosis which is a fancy term for ...
user avatar
  • 385
2 votes
1 answer
1k views

What is the difference betwen a time non-homogenous Markov Chain and a non-linear Markov Chain? Example

A time non-homogenous Markov Chain is one in which the transition probabilities are not constant over time. A non-linear Markov Chain is a model that is not linear in parameters and satisfies the ...
user avatar
1 vote
0 answers
552 views

Irreducible Markov chain and transition matrix

We know that a matrix is irreducible if it is not similar via a permutation to a block upper triangular matrix. Is the transition matrix of a irreducible Markov chain irreducible?
user avatar
1 vote
0 answers
210 views

Bayesian inference of parameter governing Markov transition matrix

A 3-state Markov chain $X = \{x_i : i \in \{1, \cdots, N\}\}$ is observed, and its transition matrix $P$ is assumed to be of the form $$ \begin{pmatrix} (1-a)^2 & 2a(1-a) & a^2 \\ b(1-a) &...
user avatar
1 vote
0 answers
137 views

Bayesian inference of non-homogeneous Markov transition matrix

The data consists of several discrete-time Markov chains, indexed by a global time. I assume all the chains are governed by the same transition matrix, but that this can change in time. I want to ...
user avatar
2 votes
0 answers
426 views

A Hidden Markov model with covariates in the transition probabilities

I would like to construct a Hidden Markov model with data about online customer journeys. A well-known concept related to the customer journey literature is the sales funnel. Consumers walk through ...
user avatar
0 votes
0 answers
83 views

How two transition probability matrices can be used to find one variable in analysis

I'm working on a research paper for cricket analysis but I'm stuck on the point of the estimation of $\tau_{owj}$, which is defined in the appendix. I have calculated up through $\alpha_{owj}$ and $\...
user avatar
0 votes
0 answers
940 views

Second order markov tranision probability matrix

I tried to find the second order Markov chain of the following sequence Dat= A A B A B A A A B B A A B I tried it on "Markov chain" package in R. ...
user avatar
0 votes
2 answers
837 views

Markov-Chain transition probabilities for 3 variables

I am a bit confused as I need to calculate the Markov-Chain transition probabilites for 3 variables. Example data, let's assume a sequence of letters at specific and progressively-constant time steps:...
user avatar
  • 284
0 votes
2 answers
1k views

Seeking examples of latent transition analysis with ordinal data and multiple subjects

Background A colleague of mine has asked me for help. She has a large amount of patient data involving clinical psychological measures (e.g., questions related to specific symptoms) gathered both ...
user avatar
  • 392