# Questions tagged [markov-process]

A stochastic process with the property that the future is conditionally independent of the past, given the present.

864 questions
196 views

### Sampler method to choose in Monte Carlo Markov chain estimation

When estimating the posterior via MCMC, are there guidelines on the best sampling method to use depending on the nature of the model? There are a variety of forms of MCMC - the Gibbs sampler, the ...
826 views

### How can I generate correlated timeseries made up of 0s and 1s?

I want to generate series of 0s and 1s that exhibit some clustering. By this I mean that 1s and 0s should occur together. So I envisage series of 0s and 1s that will exhibit similar clustering of ...
2k views

### Do we have a problem of “pity upvotes”?

I know, this may sound like it is off-topic, but hear me out. At Stack Overflow and here we get votes on posts, this is all stored in a tabular form. E.g.: post id voter id vote type ...
1k views

### Steady state probabilities for a continuous-time Markov chain

I have a finite state and time-homogeneous continuous-time Markov chain (CTMC) which is not irreducible. Will steady state probabilities exist for this CTMC? How to prove this?
748 views

### Discrepancy measures for transition matrices

I'm doing some work on modelling transition matrices, and for this I need a measure of discrepancy or lack of fit: that is, if I have a matrix $T$ and a target matrix $T_0$, I want to be able to ...
448 views

### Can we get confidence intervals for entries in stationary vector for an absorbing, time-independent Markov chain?

I have a finite-state, time-independent Markov chain with two absorbing states which models educational outcomes (the absorbing states are completing and not completing). The transition probabilities ...
688 views

### First passage time distribution in a irreducible transient discrete-time Markov chain (DTMC)

In a Markov chain, a state $j$ is transient if $f_{jj}<1$ ($f_{jj}$ is probability of ever visiting state $j$ starting from state $j$ ). Suppose, I have an irreducible transient DTMC (means all ...
333 views

### Uncertainty of conditional probability evaluated from sample

I'm doing some some analysis of an arbitrary string of text, modelling it as a Markov chain where the state is simply the value of the previous character. Call the current character ...
2k views

### Using autocorrelation to find commonly occurring signal fragments

I have a sensor which is capturing accelerometer data as a person walks. What I'm interested in extracting is each signal fragment when a step is taken. The Z-axis is what is used since only one axis ...
12k views

### Resources for learning Markov chain and hidden Markov models

I am looking for resources (tutorials, textbooks, webcast, etc) to learn about Markov Chain and HMMs. My background is as a biologist, and I'm currently involved in a bioinformatics-related project. ...
11k views

### Markov Process about only depending on previous state

I would just like someone to confirm my understanding or if I'm missing something. The definition of a markov process says the next step depends on the current state only and no past states. So, let'...
3k views

### How should one approch Project Euler problem 213 (“Flea Circus”)?

I would like to solve Project Euler 213 but don't know where to start because I'm a layperson in the field of Statistics, notice that an accurate answer is required so the Monte Carlo method won't ...