Questions tagged [transition-matrix]

A transition matrix is a square matrix used to describe the transitions of a Markov chain.

Filter by
Sorted by
Tagged with
0
votes
0answers
6 views

How to make the R code for generating Transition probability matrix from a vector of observation more precise? [closed]

This is my r code for share market close value. I have written code for the increase and decrease values. ...
0
votes
0answers
14 views

Why does an estimated transition matrix have a positive entry for a non-allowed transition? (MSM package)

I have estimated a Markov model for panel data using the R package msm to calculate the transition matrix between states. This Markov model was estimated using an ...
0
votes
0answers
13 views

Estimating a transition matrix from irregular time series when time is not of interest

I have a dataset with the following variables: id, representing person id; state_nr, representing the state that someone is in; ...
0
votes
0answers
14 views

Using Markov Chains to predict market share for prescription drugs

I wish to understand if the approach I am researching holds up to analytical rigor. I am planning to use historical data of 4 prescription drugs to forecast the market share of the same 4 drugs some ...
0
votes
0answers
16 views

Rankings transition matrix

I have heard of ratings transition matrices in finance, where it is possible to derive the probability of moving from one rating to another in a Markov chain process. If I have N individuals who are ...
0
votes
1answer
39 views

Transition Matrix for a non-trivial example

So I've just been introduced to Transition Matrices; and I was wondering what one for look like the following example: "Symmetric random walk on the integers" $S$ $=$ $\mathbb{Z}$, $\forall$ ...
1
vote
0answers
17 views

"Forcing" equal probabilities in the matrix exponential of a Markov intensity matrix

I have an upper-right triangular transition intensity matrix $Q$ for a 7-state Markov model (with states $X_1,X_2,...,X_7$), from which I numerically calculate the matrix exponential to derive a ...
0
votes
0answers
27 views

"Invalid Parent Values' in Multivariate Normal approximation of Multinomial (JAGS)

I have a transition matrix, describing the probability of an entity moving from one state to another in a time-period. I use this transition matrix to generate a series of "flow matrices", $...
0
votes
0answers
17 views

pykalman library somehow changes shape of state mean while updating KalmanFilter

I am trying to use pykalman to apply KalmanFilter on a data. When I wrote the code for KalmanFilter on my own, it was working fine but I wanted to use the ...
0
votes
0answers
22 views

How to generate a realization from a transition matrix?

Consider a Markov chain of 4 states described by the transition matrix, $$ T_{ij} = \begin{bmatrix} 0.40 & 0.56 & 0.03 & 0.01\\ 0.45 & 0.51 & 0.04 & 0.00\\ 0.25 & 0.25 &...
2
votes
2answers
271 views

Probability of doing a specific Path in a Markov Chain

My problem is the following: I have this graph, representing a Markov Chain: For example, if I am in state 1, the probability of going in state 2 or 4 is $\frac{1}{2}$. So I'm saying that the ...
0
votes
0answers
21 views

Proof: how to prove that the inverse of the fundamental matrix N of an absorbing chain exists (i.e. $N^{-1}$ exists) and $Q=I-N^{-1}$?

I know that I-Q can not have a zero determinant, so it has an inverse, i.e. $N=(I-Q)^{-1}$ exists. I think I know how to prove part b of this question given that we assume $N^{-1}$ exists, my ...
0
votes
1answer
27 views

How to derive transition matrix in this stochastic process?

I am new to stochastic processes and trying to solve a question related to finding a transition matrix of some experiment. The question is a A sequence of experiments is performed, in each of which ...
0
votes
0answers
9 views

Why doesn’t two ways of calculating stationary distribution result in the same answer?

I use matpow=function(M,n){ ans=M for(i in 1:(n-1)){ ans=ans%*%M } ans } to set the matpow function. then I enter the transition matrix ...
0
votes
0answers
24 views

Hidden Markov Model

For part b, would the answer be p = 7/10 since the left hand is biased, we would look at every q that has L and check the observation for every L that has H?
1
vote
1answer
223 views

Number of states in HMM

I am testing a HMM model by generating data from a 3x3 transition matrix and 3x4 emission matrix and then trying to train a HMM model against this data with different initializations. When I plot the ...
1
vote
0answers
40 views

How to maximize the steady state transition probability for a state in a Markov chain by altering that state's outgoing transition probabilities?

Let's say we have a transition matrix of which can be solved to come up with steady state transition probabilities of Alpha: 34.9% Beta: 24.0% Gamma: 16.9% Delta: 24.2% Now, imagine Alpha, Beta, ...
0
votes
0answers
32 views

Simulate discrete state space CTMC from generator matrix

Consider a generator matrix $Q\in\mathbb{R}^{h\times h}$ for a discrete state space $\{1,...,h\}$. I want to determine the probability of a single transition of a stochastic process $X(t)$ with $X(0)=...
0
votes
2answers
73 views

How to calculate the probability Matrix (Alpha) for Regular Markov chains

Pardon me for being a novice here. In the image attached, eq 3.1 represents the transition matrix (it's pretty clear). I am not able to comprehend the eq 3.2, alpha*P = alpha, as well as the further ...
0
votes
1answer
127 views

$\pi_i P^n_{i, j} =$ long-run proportion of time the chain is in $i$ and will be in $j$ after $n$ transitions?

I am currently studying the textbook Introduction to Probability Models by Sheldon M. Ross. Chapter 4.4 Long-Run Proportions and Limiting Probabilities says the following: Because $\pi_i$ is the long-...
0
votes
1answer
213 views

generate a realization from a transition matrix

Consider a markov chain of 4 states $\{S_1, S_2, S_3, S_4\}$ described by the transition matrix $$ A = \begin{bmatrix} .25 & .20 & .25 & .30 \\ .20 & .30 & .25 & .30 \\ ....
1
vote
1answer
26 views

Calculating limit law for matrix

I my notes on Markov chains, I am presented with the following matrix: $$\mathcal{P} = \begin{bmatrix} 0.97 & 0.03 & 0 & 0 \\ 0.008 & 0.982 & 0.01 & 0 \\ 0.02 & 0 & 0....
1
vote
1answer
59 views

Example where unique stationary law, which is an occupation law, but no limit law exists

I am currently learning about the balance equations, mass equation, limit law, occupation law and stationary law in Markov models. The following example is presented: Example 2: $$\mathcal{P} = \...
1
vote
1answer
28 views

Computation of balance equation example in Markov model

I am studying some examples of balance equations for Markov models. I am presented with the following example: $$\mathcal{P} = \begin{bmatrix} 0.2 & 0.3 & 0.5 \\ 0.1 & 0 & 0.9 \\ 0.55 ...
1
vote
1answer
75 views

Markov models and occupation time

I'm presented with the following explanation and proof: Let $(X_n)$ be a Markov chain, and fix a state $j \in S$. Define indicator variables: For $n = 0, 1, \dots$, let $$I_n(j) = \begin{cases} 1 &...
1
vote
1answer
174 views

Calculating 7-step transition matrix for example

In my notes on Markov processes, I am presented with two related examples: Example 1: Classify daily weather for some region as Sunny (state $1$), Cloudy (state $2$), or rainy (state $3$). ...
0
votes
1answer
259 views

Proof that the $n$-step transition matrix is the $n$th power of $\mathcal{P}$

I am presented with the following theorem in the context of Markov chains and stochastic systems: The $n$-step transition matrix is the $n$th power of $\mathcal{P}$: $$\mathcal{P}^{(n)} = P^n.$$ And ...
1
vote
0answers
17 views

Transition probability matrix

A gambler tosses a coin and a tetrahedron at each stage. If $H$, he receives the amount appearing at the face of the tetrahedron. If $T$, he pays the amount. The tetrahedron is fair, but probability ...
0
votes
0answers
16 views

Find transition probability matrix

A box contains 3 balls. Each is either white or red. The game is to draw a ball from each period. If red is drawn, a white is replaced. But if white is drawn, all the balls in the box is replaced by ...
0
votes
1answer
68 views

Number of stationary distributions of a Markov chain

How do i determine the number of stationary distributions that a Markov chain has if it is not irreducible or regular. The transition matrix is ...
2
votes
1answer
296 views

Estimating model for transition probabilities of a Markov Chain

Suppose that I have a Markov chain with $S$ states evolving over time. I have $S^2\times T$ values of the transition matrix, where $T$ is the number of time periods. I also have $K$ matrices $X$ of $T\...
4
votes
1answer
150 views

How to show that the transition probability is equal to $\overline p_{ij} = \frac{P_{ij}}{\sum_{k\neq i}p_{ik}}$

(No new answers needed) I would like to award @whuber for his good answer with my bounty! Suppose that $(X_n)_{n≥0}$ is Markov$(λ, P)$ but that we only observe the process when it moves to a new ...
1
vote
1answer
302 views

Rows and columns of the one-step transition probability matrix

I am currently studying the textbook Introduction to Modeling and Analysis of Stochastic Systems, Second Edition, by V. G. Kulkarni. In a section on discrete-time Markov chains, the author introduces ...
0
votes
0answers
39 views

Singular state transition probability matrix in David Silver's UCL Lesson 2

I'm studying David Silver's second lesson on reinforcement learning: https://www.youtube.com/watch?v=lfHX2hHRMVQ&list=PLqYmG7hTraZDM-OYHWgPebj2MfCFzFObQ&index=2 and the state transition ...
1
vote
0answers
20 views

Specifying a multi-state model with unobservable terminal state

Suppose there is a multi-state process with three states, listed below and labelled as terminal/non-terminal and observable/unobservable: Initializing: non-terminal observable Active: non-terminal ...
0
votes
0answers
187 views

Make sense of plotting a transition matrix

I'm studying statistics and I'm trying to understand markov chain topic. I'm using the package "markovchain" in R to obtain the stationary distribution. From this transition matrix $M$: <...
2
votes
1answer
535 views

Measure the distance between two probability transition matrices

I have a probability transition matrix $P$ that contains some values very close to zero. I want to sparsify this matrix by taking the k largest values for each row and setting the others to zero. For ...
0
votes
1answer
60 views

Long run proportion of transitions in a Markov chain

Let $S$ be a set of states for a Markov chain and let $S^C$ be the remaining states. Explain the identity $$\sum_{i\in S}\sum_{j\in S^C}\pi_iP_{ij}=\sum_{i\in S^C}\sum_{j\in S}\pi_iP_{ij}$$ I know ...
1
vote
1answer
365 views

Irreducible (communicating) classes [closed]

The Markov chain $(Xn; n\geq)$ has state-space $S = (0, 1, 2, . . .)$, with $p_{i,0} = \frac{1}{4}$ and $p_{i,i+1} = \frac{3}{4}$ $\forall i \geq 0$, so that the transition matrix is P =$\...
0
votes
1answer
305 views

Biased coins and Markov processes

Good day, I am attempting an optional exercise and I am finding it hard to interpret the problem in terms of matrices and vectors. Coin 1 has probability 0.4 of coming up heads, and coin 2 has ...
0
votes
0answers
101 views

Formulating a Transition matrix for Markov Process

I am dealing with a medical process which is as follows. There are 10000 Veterans who are enrolled in this study. All 10000 have medical condition called onychocryptosis which is a fancy term for ...
2
votes
1answer
596 views

What is the difference betwen a time non-homogenous Markov Chain and a non-linear Markov Chain? Example

A time non-homogenous Markov Chain is one in which the transition probabilities are not constant over time. A non-linear Markov Chain is a model that is not linear in parameters and satisfies the ...
1
vote
0answers
494 views

Irreducible Markov chain and transition matrix

We know that a matrix is irreducible if it is not similar via a permutation to a block upper triangular matrix. Is the transition matrix of a irreducible Markov chain irreducible?
1
vote
0answers
188 views

Bayesian inference of parameter governing Markov transition matrix

A 3-state Markov chain $X = \{x_i : i \in \{1, \cdots, N\}\}$ is observed, and its transition matrix $P$ is assumed to be of the form $$ \begin{pmatrix} (1-a)^2 & 2a(1-a) & a^2 \\ b(1-a) &...
1
vote
0answers
128 views

Bayesian inference of non-homogeneous Markov transition matrix

The data consists of several discrete-time Markov chains, indexed by a global time. I assume all the chains are governed by the same transition matrix, but that this can change in time. I want to ...
2
votes
0answers
379 views

A Hidden Markov model with covariates in the transition probabilities

I would like to construct a Hidden Markov model with data about online customer journeys. A well-known concept related to the customer journey literature is the sales funnel. Consumers walk through ...
0
votes
0answers
82 views

How two transition probability matrices can be used to find one variable in analysis

I'm working on a research paper for cricket analysis but I'm stuck on the point of the estimation of $\tau_{owj}$, which is defined in the appendix. I have calculated up through $\alpha_{owj}$ and $\...
0
votes
0answers
750 views

Second order markov tranision probability matrix

I tried to find the second order Markov chain of the following sequence Dat= A A B A B A A A B B A A B I tried it on "Markov chain" package in R. ...
0
votes
2answers
658 views

Markov-Chain transition probabilities for 3 variables

I am a bit confused as I need to calculate the Markov-Chain transition probabilites for 3 variables. Example data, let's assume a sequence of letters at specific and progressively-constant time steps:...
0
votes
2answers
1k views

Seeking examples of latent transition analysis with ordinal data and multiple subjects

Background A colleague of mine has asked me for help. She has a large amount of patient data involving clinical psychological measures (e.g., questions related to specific symptoms) gathered both ...