Questions tagged [transition-matrix]
A transition matrix is a square matrix used to describe the transitions of a Markov chain.
108 questions
2
votes
1
answer
1k
views
Measure the distance between two probability transition matrices
I have a probability transition matrix $P$ that contains some values very close to zero. I want to sparsify this matrix by taking the k largest values for each row and setting the others to zero. For ...
- 111
0
votes
1
answer
124
views
Long run proportion of transitions in a Markov chain
Let $S$ be a set of states for a Markov chain and let $S^C$ be the remaining states. Explain the identity $$\sum_{i\in S}\sum_{j\in S^C}\pi_iP_{ij}=\sum_{i\in S^C}\sum_{j\in S}\pi_iP_{ij}$$
I know ...
- 1
1
vote
1
answer
591
views
Irreducible (communicating) classes [closed]
The Markov chain $(Xn; n\geq)$ has state-space $S = (0, 1, 2, . . .)$, with
$p_{i,0} = \frac{1}{4}$ and $p_{i,i+1} = \frac{3}{4}$ $\forall i \geq 0$, so that the transition matrix is
P =$\...
1
vote
2
answers
816
views
Biased coins and Markov processes
Good day, I am attempting an optional exercise and I am finding it hard to interpret the problem in terms of matrices and vectors.
Coin 1 has probability 0.4 of coming up heads, and coin 2 has ...
0
votes
0
answers
117
views
Formulating a Transition matrix for Markov Process
I am dealing with a medical process which is as follows.
There are 10000 Veterans who are enrolled in this study.
All 10000 have medical condition called onychocryptosis which is a fancy term for ...
- 577
2
votes
1
answer
3k
views
What is the difference betwen a time non-homogenous Markov Chain and a non-linear Markov Chain? Example
A time non-homogenous Markov Chain is one in which the transition probabilities are not constant over time. A non-linear Markov Chain is a model that is not linear in parameters and satisfies the ...
- 4,792
2
votes
0
answers
675
views
Irreducible Markov chain and transition matrix
We know that a matrix is irreducible if it is not similar via a permutation to a block upper triangular matrix.
Is the transition matrix of a irreducible Markov chain irreducible?
- 21
1
vote
0
answers
259
views
Bayesian inference of parameter governing Markov transition matrix
A 3-state Markov chain $X = \{x_i : i \in \{1, \cdots, N\}\}$ is observed, and its transition matrix $P$ is assumed to be of the form
$$
\begin{pmatrix}
(1-a)^2 & 2a(1-a) & a^2 \\
b(1-a) &...
1
vote
0
answers
244
views
Bayesian inference of non-homogeneous Markov transition matrix
The data consists of several discrete-time Markov chains, indexed by a global time. I assume all the chains are governed by the same transition matrix, but that this can change in time. I want to ...
2
votes
0
answers
624
views
A Hidden Markov model with covariates in the transition probabilities
I would like to construct a Hidden Markov model with data about online customer journeys. A well-known concept related to the customer journey literature is the sales funnel. Consumers walk through ...
0
votes
0
answers
89
views
How two transition probability matrices can be used to find one variable in analysis
I'm working on a research paper for cricket analysis but I'm stuck on the point of the estimation of $\tau_{owj}$, which is defined in the appendix. I have calculated up through $\alpha_{owj}$ and $\...
0
votes
0
answers
1k
views
Second order markov tranision probability matrix
I tried to find the second order Markov chain of the following sequence
Dat= A A B A B A A A B B A A B
I tried it on "Markov chain" package in R.
...
0
votes
2
answers
1k
views
Markov-Chain transition probabilities for 3 variables
I am a bit confused as I need to calculate the Markov-Chain transition probabilites for 3 variables.
Example data, let's assume a sequence of letters at specific and progressively-constant time steps:...
- 344
2
votes
3
answers
3k
views
Seeking examples of latent transition analysis with ordinal data and multiple subjects
Background
A colleague of mine has asked me for help. She has a large amount of patient data involving clinical psychological measures (e.g., questions related to specific symptoms) gathered both ...
- 432
0
votes
1
answer
2k
views
MLE for Markov Chains - intuitive explanation
could anyone please give me intuitive explanation what does below mean ?
Let say I have sequence:
a, b, a, b, b, b, a, b, b, a
By applying Markov Chains with MLE method in R package on below I get ...
0
votes
1
answer
396
views
Irreducible Markov Chain Question
If you have an irreducible Markov chain with transition matrix $P$, and $p(j,j) > 0$ for all $j$, why are all states aperiodic?
- 1
1
vote
1
answer
2k
views
"Symmetric" property of stationary distribution
The above symmetric property isn't referring to the double stochastic property
\begin{bmatrix}
0.2&0.8&0&0&0\\
0.2&0.2&0.6&0&0\\
0&0.4&0.2&0.4&0\\
0&...
- 11
1
vote
1
answer
2k
views
Expected value in Markov chains
Let $\left\{X_{n}\right\}_{n\geq0}$ be a homogeneous markov chain with state space E and transition matrix P. Let $\tau$ be the first time n for which $X_{n}$ $\neq$ $X_{0}$, where $\tau=+\infty$ if $...
- 543
1
vote
0
answers
71
views
How to find certain probabilities in transition matrices
A unit consists of three circuit elements in parallel; it will be in working order as long as at least one of the elements is working. The unit is examined every hour and if it is not in working order ...
- 11
0
votes
0
answers
225
views
How to bet on a binary event based on the markov transition matrix, state probabilities and the odds
There is a coupon full of football matches for a given day from a bookkeeper. I have scrapped another website and i have aquired continuous history of a particular match between ...
- 141
4
votes
1
answer
7k
views
How to translate an adjacency matrix to a transition matrix for use in Markov cluster algorithm? [closed]
I have a matrix of size (47*47 double) that have only 0's and 1's.
I want to apply the Markov clustering algorithm on this matrix, but this Method needs a transition matrix as the columns must be ...
- 93
5
votes
0
answers
778
views
Find the invariant measure $\pi=(\pi_{1},\pi_{2},\pi_{3})$ for a Markov Chain with transition matrix given
Let $(X_{n})_{n\in\mathbb{N}_{0}}$ be a Markov Chain with state space $M=\left\{x_{1},x_{2},x_{3}\right\}$ and transtition matrix
$$
\Pi=\left(\begin{array}{ccc}p_{1} & p_{2} & 1-p_{1}-p_{2}\\
...
- 491
0
votes
0
answers
37
views
Calculate the transition matrix of $X_{n+1}:= \sum_{i=0}^{X_{n}}\theta_{n}^{i}\:\: \mbox{mod }5.$ where $\theta_{n}^{i}\sim Bin(3,1/3)$ i.i.d
Given an i.i.d. sequence $(\theta_{n}^{i})_{n,i\in\mathbb{N}}$ with binomial distribution $\mathcal{B}_{3,\frac{1}{3}}$ we define the Markov Chain
$$X_{n+1}:= \sum_{i=0}^{X_{n}}\theta_{n}^{i}\:\:\: \...
- 491
5
votes
1
answer
2k
views
Constructing a transition probability from Q-learning
In Reinforcement learning, learning without the need for the transition probability matrix is 'model free learning'. Instead of having the transition probabilities, we learn the q-values (state/action ...
- 9,317
1
vote
0
answers
38
views
Markov process with conditions
I am trying to model a basketball game using play by play data. Each team has a transition matrix representing what they will do next on offense and defense in any given time remaining in the game.
...
- 21
3
votes
1
answer
1k
views
Finding steady-state probability of a Markov chain
Let $X_{n}$ be a Markov chain on state space $S = \{ 1,2 \dots, 23 \}$ with transition probability given by
$p_{i,i+1}= p_{i,i-1} = \frac {1}{2} \ \ \forall \ 2\le i \le 22 , $
$ p_{1,2}= p_{1,23} =...
- 663
4
votes
4
answers
5k
views
Essential transient state in a Markov chain
Can a finite state Markov chain have essential transient state?
I have found out an example for an infinite state one and I have the intuition (I may be wrong) that for a finite state space .. This ...
- 1,267
6
votes
1
answer
2k
views
The expected long run proportion of time the chain spends at $a$ , given that it starts at $c$
Consider the transition matrix:
$\begin{bmatrix} \frac{1}{5} & \frac{4}{5} & 0 & 0 & 0 \\
\frac{1}{2} & \frac{1}{2} & 0 & 0 & 0 \\ \frac{1}{5} & \frac{1}{5} & \...
- 385
0
votes
0
answers
136
views
Stationary Probabilities
A transition probability matrix $P$ is said to be doubly stochastic, if each column sum is 1;that is if
$\displaystyle\sum_i P_{i,j} = 1$ for all j
If such a chain is irreducible and have states $1,...
- 1,226
10
votes
0
answers
353
views
Reinforcement *Model* Learning
Classical reinforcement learning (Q- or Sarsa-Learning) can be extended with models of the environment. These models are usually transition tables that contain the probability of arriving at a ...
- 546
1
vote
1
answer
833
views
Regular Transition Matrix
A transition matrix is regular if some power of the matrix contains
all positive entries. [1]
Its powers have all positive entries...
Why isn't this matrix a Regular Transition Matrix?
Reference:
...
- 2,125
1
vote
0
answers
340
views
how to calculate transition probability matrix for n*4 data
data: https://www.dropbox.com/s/4ni96jmzn14sjl4/florep_normn4_p1.mat?dl=0
I attached n*4 data. so here each row has 4 values and each row considered as one state. I want to calculate transition ...
- 11
1
vote
1
answer
1k
views
Transition matrix calculation from text book "Introduction to information retrieval"
According to the example on page in the text book, the graph of documents is as follows:
$1\rightarrow2, 3\rightarrow2, 2\rightarrow1, 2\rightarrow3$ with $\lambda = 0.5$ which would form an ...
- 111
3
votes
1
answer
1k
views
Markov chain transition matrix and state vector order?
The typical way of performing state simulations using matrices is to perform the following kind of calculation:
$$x_{(k+1)}=Ax_{(k)}$$
However,
When the transition matrix $A$ is defined in the ...
- 4,129
0
votes
1
answer
2k
views
Markov Chain for system with failure and repair
I am solving the below problem, and facing a few doubts regarding my approach.
Consider a system with two components. We observe the state of the system every hour. A given component operating at ...
- 103
2
votes
1
answer
2k
views
Use multi-state modelling (MSM) to predict individual transition probabilities in R?
I am using R package msm to model multi-state transitions in the sample dataset cav:
...
- 43
1
vote
1
answer
227
views
transition matrix for urn model
There are slides regrading to urn model
I have two questions
if a Species A dies and a Species A is born, the original text says the probability is 0.4*0.4, but since a Species A has died , only ...
- 757
1
vote
1
answer
517
views
Randomly generating transition probabilities for Markov chains
I'm trying to simulate a person moving through a household using a Markov chain. Each state would be a room in the house. The issue I'm running into is that I have no existing data telling me what a ...
1
vote
0
answers
165
views
'Lumpable states' analysis for a large transition matrix
I have a large transition matrix, whereby I calculate n-step state distribution results for n=1..10, and then merge states of interest for each ...
- 155
4
votes
2
answers
1k
views
Transition probabilities - Markov chains
I have a homogeneous Markov chain with transition matrix
I want to compute $P(Y_1 = 1| Y_2=2)$ where $Y_t, t=1,2$ is the observation at time $t$ and $Y_0=3$. I tried with Bayes' rule, so
$$P(Y_1 = 1|...
- 71
6
votes
1
answer
4k
views
Significance testing for Markov chain transition probabilities
The question
How can I calculate p values for individual transitions in a Markov chain? I want to test the null hypothesis that the probability of entering state $B$ from previous state $A$ is less ...
- 33.2k
2
votes
1
answer
191
views
What can I do with NA values in my second-order Markov chain?
I have states A, B, C, I have developed both a 1st and 2nd Order Markov Chain for them.
Each state represents a status that an individual can be in, and the transitions represents the probability of ...
- 391
1
vote
1
answer
2k
views
Why is it necessary to fix a matrix diagonal and after this calculate the exponential to assess transition probabilities?
I'm learning markov chains in order to compute estimations of transition probabilities, and I found an example of the estimator construction for continuous time markov chains:
http://www.rinfinance....
- 123
0
votes
1
answer
2k
views
Transition probability matrix rows not summing to 1
I'm implementing a method of sampling from a CTMC from here.
I'm trying to calculate the transition probability matrix but the rows are not summing to 1, except at 0.
I've diagonalised my rate ...
- 175
1
vote
0
answers
332
views
Are transition matrices and markov chains considered a part of time series analysis
Is the topic of Markov Process and Transition Matrices considered a part of time series analysis? Transition matrix has the intuition of being a time based process because the transition probabilities ...
- 6,635
2
votes
1
answer
225
views
What kind of questions can I answer if I have a transition matrix of a system
I understand how a transition matrix looks. It is a matrix of probabilities of moving between one state to another. After I am given such a matrix, what are some questions that I can answer about the ...
- 6,635
0
votes
1
answer
266
views
How to report simple markov transition matrix
I have a beginners question on Markov chains. We found that a Markov chain might be a good way to describe the data we got from our experiment. Let's say we have two simple Markov transition matrix ...
- 205
-1
votes
1
answer
417
views
How to Compute for the Transition Probability Matrix F
this is our homework. I don't understand how to compute for the F matrix...I'm having a hard time understanding this transition probability matrix. Thank you in advance!
0
votes
1
answer
1k
views
Transition Probability Matrix with zeros
A markov chain model TPM for daily rainfall is given by:
[0.4 0.6 0.0;
0.2 0.6 0.2;
0.0 0.7 0.3];
Thus, it is not possible to pass directly from state 1 to ...
- 103
4
votes
1
answer
508
views
First-order Discrete Markov Chain with time lag
I want to estimate the first-order transition matrix of a sequence in discrete time, e.g.
$$ s = 1,0,1,0,1,1,0,1,0,0, \dots$$
but states are not evenly spaced in time. So that even if $s_{t=1} = 1$ ...
- 5,156