Questions tagged [transition-matrix]

A transition matrix is a square matrix used to describe the transitions of a Markov chain.

Filter by
Sorted by
Tagged with
0
votes
0answers
10 views

Using Markov Chains to predict market share for prescription drugs

I wish to understand if the approach I am researching holds up to analytical rigor. I am planning to use historical data of 4 prescription drugs to forecast the market share of the same 4 drugs some ...
0
votes
0answers
10 views

Rankings transition matrix

I have heard of ratings transition matrices in finance, where it is possible to derive the probability of moving from one rating to another in a Markov chain process. If I have N individuals who are ...
0
votes
1answer
39 views

Transition Matrix for a non-trivial example

So I've just been introduced to Transition Matrices; and I was wondering what one for look like the following example: "Symmetric random walk on the integers" $S$ $=$ $\mathbb{Z}$, $\forall$ ...
1
vote
0answers
12 views

“Forcing” equal probabilities in the matrix exponential of a Markov intensity matrix

I have an upper-right triangular transition intensity matrix $Q$ for a 7-state Markov model (with states $X_1,X_2,...,X_7$), from which I numerically calculate the matrix exponential to derive a ...
0
votes
0answers
15 views

"Invalid Parent Values' in Multivariate Normal approximation of Multinomial (JAGS)

I have a transition matrix, describing the probability of an entity moving from one state to another in a time-period. I use this transition matrix to generate a series of "flow matrices", $...
0
votes
0answers
7 views

pykalman library somehow changes shape of state mean while updating KalmanFilter

I am trying to use pykalman to apply KalmanFilter on a data. When I wrote the code for KalmanFilter on my own, it was working fine but I wanted to use the ...
0
votes
0answers
19 views

How to generate a realization from a transition matrix?

Consider a Markov chain of 4 states described by the transition matrix, $$ T_{ij} = \begin{bmatrix} 0.40 & 0.56 & 0.03 & 0.01\\ 0.45 & 0.51 & 0.04 & 0.00\\ 0.25 & 0.25 &...
2
votes
2answers
193 views

Probability of doing a specific Path in a Markov Chain

My problem is the following: I have this graph, representing a Markov Chain: For example, if I am in state 1, the probability of going in state 2 or 4 is $\frac{1}{2}$. So I'm saying that the ...
0
votes
0answers
20 views

Proof: how to prove that the inverse of the fundamental matrix N of an absorbing chain exists (i.e. $N^{-1}$ exists) and $Q=I-N^{-1}$?

I know that I-Q can not have a zero determinant, so it has an inverse, i.e. $N=(I-Q)^{-1}$ exists. I think I know how to prove part b of this question given that we assume $N^{-1}$ exists, my ...
0
votes
1answer
25 views

How to derive transition matrix in this stochastic process?

I am new to stochastic processes and trying to solve a question related to finding a transition matrix of some experiment. The question is a A sequence of experiments is performed, in each of which ...
0
votes
0answers
7 views

Why doesn’t two ways of calculating stationary distribution result in the same answer?

I use matpow=function(M,n){ ans=M for(i in 1:(n-1)){ ans=ans%*%M } ans } to set the matpow function. then I enter the transition matrix ...
0
votes
0answers
22 views

Hidden Markov Model

For part b, would the answer be p = 7/10 since the left hand is biased, we would look at every q that has L and check the observation for every L that has H?
1
vote
1answer
104 views

Number of states in HMM

I am testing a HMM model by generating data from a 3x3 transition matrix and 3x4 emission matrix and then trying to train a HMM model against this data with different initializations. When I plot the ...
0
votes
0answers
28 views

Statistically compare transition probabilities from two transition matrices

I have a dataset that is stratified by gender. I have fit a first-order Markov chain to both strata, giving transition matrices $M_1$ and $M_2$. Now I want to statistically test, for given $i, j$, ...
1
vote
0answers
29 views

How to maximize the steady state transition probability for a state in a Markov chain by altering that state's outgoing transition probabilities?

Let's say we have a transition matrix of which can be solved to come up with steady state transition probabilities of Alpha: 34.9% Beta: 24.0% Gamma: 16.9% Delta: 24.2% Now, imagine Alpha, Beta, ...
0
votes
0answers
31 views

Simulate discrete state space CTMC from generator matrix

Consider a generator matrix $Q\in\mathbb{R}^{h\times h}$ for a discrete state space $\{1,...,h\}$. I want to determine the probability of a single transition of a stochastic process $X(t)$ with $X(0)=...
0
votes
0answers
21 views

Creating a transition matrix based on a Markov chain in R

I have four distributions that represent incomes in R. I categorise them by what income group they fall under such as under half the mean, between half the mean and 3/4th of the mean and so on until ...
0
votes
0answers
13 views

Transition probablities vs transition rate

What would you describe and differentiate between Transition rate and probabilities intuitively in accordance with Transition probablity matrix as well as markov chains and HSD models.
0
votes
2answers
39 views

How to calculate the probability Matrix (Alpha) for Regular Markov chains

Pardon me for being a novice here. In the image attached, eq 3.1 represents the transition matrix (it's pretty clear). I am not able to comprehend the eq 3.2, alpha*P = alpha, as well as the further ...
0
votes
0answers
61 views

Accuracy in Estimating Customer Lifetime Value using Markov Chain Model

I've an online customer data which has the purchases made in every month and recency of the purchases information for 12 months. So data looks like below: ...
0
votes
1answer
101 views

$\pi_i P^n_{i, j} =$ long-run proportion of time the chain is in $i$ and will be in $j$ after $n$ transitions?

I am currently studying the textbook Introduction to Probability Models by Sheldon M. Ross. Chapter 4.4 Long-Run Proportions and Limiting Probabilities says the following: Because $\pi_i$ is the long-...
0
votes
1answer
96 views

generate a realization from a transition matrix

Consider a markov chain of 4 states $\{S_1, S_2, S_3, S_4\}$ described by the transition matrix $$ A = \begin{bmatrix} .25 & .20 & .25 & .30 \\ .20 & .30 & .25 & .30 \\ ....
1
vote
1answer
23 views

Calculating limit law for matrix

I my notes on Markov chains, I am presented with the following matrix: $$\mathcal{P} = \begin{bmatrix} 0.97 & 0.03 & 0 & 0 \\ 0.008 & 0.982 & 0.01 & 0 \\ 0.02 & 0 & 0....
1
vote
1answer
58 views

Example where unique stationary law, which is an occupation law, but no limit law exists

I am currently learning about the balance equations, mass equation, limit law, occupation law and stationary law in Markov models. The following example is presented: Example 2: $$\mathcal{P} = \...
1
vote
1answer
26 views

Computation of balance equation example in Markov model

I am studying some examples of balance equations for Markov models. I am presented with the following example: $$\mathcal{P} = \begin{bmatrix} 0.2 & 0.3 & 0.5 \\ 0.1 & 0 & 0.9 \\ 0.55 ...
1
vote
1answer
50 views

Markov models and occupation time

I'm presented with the following explanation and proof: Let $(X_n)$ be a Markov chain, and fix a state $j \in S$. Define indicator variables: For $n = 0, 1, \dots$, let $$I_n(j) = \begin{cases} 1 &...
1
vote
1answer
41 views

Calculating 7-step transition matrix for example

In my notes on Markov processes, I am presented with two related examples: Example 1: Classify daily weather for some region as Sunny (state $1$), Cloudy (state $2$), or rainy (state $3$). ...
0
votes
1answer
159 views

Proof that the $n$-step transition matrix is the $n$th power of $\mathcal{P}$

I am presented with the following theorem in the context of Markov chains and stochastic systems: The $n$-step transition matrix is the $n$th power of $\mathcal{P}$: $$\mathcal{P}^{(n)} = P^n.$$ And ...
1
vote
0answers
17 views

Transition probability matrix

A gambler tosses a coin and a tetrahedron at each stage. If $H$, he receives the amount appearing at the face of the tetrahedron. If $T$, he pays the amount. The tetrahedron is fair, but probability ...
0
votes
0answers
16 views

Find transition probability matrix

A box contains 3 balls. Each is either white or red. The game is to draw a ball from each period. If red is drawn, a white is replaced. But if white is drawn, all the balls in the box is replaced by ...
0
votes
1answer
47 views

Number of stationary distributions of a Markov chain

How do i determine the number of stationary distributions that a Markov chain has if it is not irreducible or regular. The transition matrix is ...
2
votes
1answer
218 views

Estimating model for transition probabilities of a Markov Chain

Suppose that I have a Markov chain with $S$ states evolving over time. I have $S^2\times T$ values of the transition matrix, where $T$ is the number of time periods. I also have $K$ matrices $X$ of $T\...
5
votes
1answer
127 views

How to show that the transition probability is equal to $\overline p_{ij} = \frac{P_{ij}}{\sum_{k\neq i}p_{ik}}$

(No new answers needed) I would like to award @whuber for his good answer with my bounty! Suppose that $(X_n)_{n≥0}$ is Markov$(λ, P)$ but that we only observe the process when it moves to a new ...
1
vote
1answer
172 views

Rows and columns of the one-step transition probability matrix

I am currently studying the textbook Introduction to Modeling and Analysis of Stochastic Systems, Second Edition, by V. G. Kulkarni. In a section on discrete-time Markov chains, the author introduces ...
0
votes
0answers
36 views

Singular state transition probability matrix in David Silver's UCL Lesson 2

I'm studying David Silver's second lesson on reinforcement learning: https://www.youtube.com/watch?v=lfHX2hHRMVQ&list=PLqYmG7hTraZDM-OYHWgPebj2MfCFzFObQ&index=2 and the state transition ...
1
vote
0answers
18 views

Specifying a multi-state model with unobservable terminal state

Suppose there is a multi-state process with three states, listed below and labelled as terminal/non-terminal and observable/unobservable: Initializing: non-terminal observable Active: non-terminal ...
0
votes
0answers
175 views

Make sense of plotting a transition matrix

I'm studying statistics and I'm trying to understand markov chain topic. I'm using the package "markovchain" in R to obtain the stationary distribution. From this transition matrix $M$: <...
1
vote
1answer
380 views

Measure the distance between two probability transition matrices

I have a probability transition matrix $P$ that contains some values very close to zero. I want to sparsify this matrix by taking the k largest values for each row and setting the others to zero. For ...
0
votes
1answer
59 views

Long run proportion of transitions in a Markov chain

Let $S$ be a set of states for a Markov chain and let $S^C$ be the remaining states. Explain the identity $$\sum_{i\in S}\sum_{j\in S^C}\pi_iP_{ij}=\sum_{i\in S^C}\sum_{j\in S}\pi_iP_{ij}$$ I know ...
1
vote
1answer
336 views

Irreducible (communicating) classes [closed]

The Markov chain $(Xn; n\geq)$ has state-space $S = (0, 1, 2, . . .)$, with $p_{i,0} = \frac{1}{4}$ and $p_{i,i+1} = \frac{3}{4}$ $\forall i \geq 0$, so that the transition matrix is P =$\...
0
votes
1answer
233 views

Biased coins and Markov processes

Good day, I am attempting an optional exercise and I am finding it hard to interpret the problem in terms of matrices and vectors. Coin 1 has probability 0.4 of coming up heads, and coin 2 has ...
0
votes
0answers
101 views

Formulating a Transition matrix for Markov Process

I am dealing with a medical process which is as follows. There are 10000 Veterans who are enrolled in this study. All 10000 have medical condition called onychocryptosis which is a fancy term for ...
2
votes
1answer
431 views

What is the difference betwen a time non-homogenous Markov Chain and a non-linear Markov Chain? Example

A time non-homogenous Markov Chain is one in which the transition probabilities are not constant over time. A non-linear Markov Chain is a model that is not linear in parameters and satisfies the ...
1
vote
0answers
472 views

Irreducible Markov chain and transition matrix

We know that a matrix is irreducible if it is not similar via a permutation to a block upper triangular matrix. Is the transition matrix of a irreducible Markov chain irreducible?
1
vote
0answers
181 views

Bayesian inference of parameter governing Markov transition matrix

A 3-state Markov chain $X = \{x_i : i \in \{1, \cdots, N\}\}$ is observed, and its transition matrix $P$ is assumed to be of the form $$ \begin{pmatrix} (1-a)^2 & 2a(1-a) & a^2 \\ b(1-a) &...
1
vote
0answers
122 views

Bayesian inference of non-homogeneous Markov transition matrix

The data consists of several discrete-time Markov chains, indexed by a global time. I assume all the chains are governed by the same transition matrix, but that this can change in time. I want to ...
2
votes
0answers
354 views

A Hidden Markov model with covariates in the transition probabilities

I would like to construct a Hidden Markov model with data about online customer journeys. A well-known concept related to the customer journey literature is the sales funnel. Consumers walk through ...
0
votes
0answers
80 views

How two transition probability matrices can be used to find one variable in analysis

I'm working on a research paper for cricket analysis but I'm stuck on the point of the estimation of $\tau_{owj}$, which is defined in the appendix. I have calculated up through $\alpha_{owj}$ and $\...
0
votes
0answers
694 views

Second order markov tranision probability matrix

I tried to find the second order Markov chain of the following sequence Dat= A A B A B A A A B B A A B I tried it on "Markov chain" package in R. ...
0
votes
2answers
573 views

Markov-Chain transition probabilities for 3 variables

I am a bit confused as I need to calculate the Markov-Chain transition probabilites for 3 variables. Example data, let's assume a sequence of letters at specific and progressively-constant time steps:...