Questions tagged [markov-process]

A stochastic process with the property that the future is conditionally independent of the past, given the present.

956 questions
Filter by
Sorted by
Tagged with
14 views

31 views

How to calculate a 𝑛-step transitions of a Discrete-time Markov Chain for Figure 17.1 (b) in book “Machine Learning - A Probabilistic Perspective”

chapter 17 of the book "Machine Learning - A Probabilistic Perspective" gives this figure which is the probability of getting from i to j in exactly n steps. Obviously A(1) = A. In the case of ...
37 views

What is the point of doing simulation on Markov Chain?

I am studying Markov Chain and I am currently reading about simulation on Markov Chain but I can't see the point of simulation on Markov Chain. What does simulation mean in Markov Chain and what can ...
27 views

can the prior probability change with time?

I was reading Markov Models for sequence modeling and stuck with my understanding(hypothesis). In Baye's theorem, can the prior probabilities change with time? If the prior probability at t=3 is 0.05,...
57 views

Measure the distance between two probability transition matrices

I have a probability transition matrix $P$ that contains some values very close to zero. I want to sparsify this matrix by taking the k largest values for each row and setting the others to zero. For ...
12 views

Markov Chain Monte Carlo for Decryption Purposes

I've been reading the "Markov Chain Monte Carlov Revolution" paper by Diaconis and was intrigued by his application to decryption. I've done some further reading and am now trying to work through ...
17 views

What are the necessary qualifications or assumptions to say that a graph structure is a Markov Chain?

I have a graph structure and want to say it is a Markov Chain. But I am wondering what necessary assumptions or properties that my graph structure need to meet to be called a Markov Chain?
6 views

What are the relationships among Markov Property, Stationarity, and Time Invariance

I am wondering if there is or are any relationship among those. I have understood Markov Property by reading Wikipedia, but it is still confusing to figure out if there is any relationship among those ...
26 views

How to train a single discrete-time markov model?

I have a training set of sequences. I want to reach a discrete time Markov model (transition probability matrix). Is there a Bayesian way other than MLE to achieve this?
61 views

Spread of number of steps to reach absorbing state in markov chain

I know how to calculate the variance of the number of steps in an absorbing markov chain. However, I am not sure that the distribution of the number of steps is normal. Therefore I would like to ...