Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [markov-process]

A stochastic process with the property that the future is conditionally independent of the past, given the present.

3
votes
1answer
196 views

Sampler method to choose in Monte Carlo Markov chain estimation

When estimating the posterior via MCMC, are there guidelines on the best sampling method to use depending on the nature of the model? There are a variety of forms of MCMC - the Gibbs sampler, the ...
5
votes
1answer
826 views

How can I generate correlated timeseries made up of 0s and 1s?

I want to generate series of 0s and 1s that exhibit some clustering. By this I mean that 1s and 0s should occur together. So I envisage series of 0s and 1s that will exhibit similar clustering of ...
50
votes
3answers
2k views

Do we have a problem of “pity upvotes”?

I know, this may sound like it is off-topic, but hear me out. At Stack Overflow and here we get votes on posts, this is all stored in a tabular form. E.g.: post id voter id vote type ...
0
votes
1answer
1k views

Steady state probabilities for a continuous-time Markov chain

I have a finite state and time-homogeneous continuous-time Markov chain (CTMC) which is not irreducible. Will steady state probabilities exist for this CTMC? How to prove this?
6
votes
2answers
748 views

Discrepancy measures for transition matrices

I'm doing some work on modelling transition matrices, and for this I need a measure of discrepancy or lack of fit: that is, if I have a matrix $T$ and a target matrix $T_0$, I want to be able to ...
6
votes
1answer
448 views

Can we get confidence intervals for entries in stationary vector for an absorbing, time-independent Markov chain?

I have a finite-state, time-independent Markov chain with two absorbing states which models educational outcomes (the absorbing states are completing and not completing). The transition probabilities ...
2
votes
1answer
688 views

First passage time distribution in a irreducible transient discrete-time Markov chain (DTMC)

In a Markov chain, a state $j$ is transient if $f_{jj}<1$ ($f_{jj}$ is probability of ever visiting state $j$ starting from state $j$ ). Suppose, I have an irreducible transient DTMC (means all ...
4
votes
2answers
333 views

Uncertainty of conditional probability evaluated from sample

I'm doing some some analysis of an arbitrary string of text, modelling it as a Markov chain where the state is simply the value of the previous character. Call the current character ...
7
votes
1answer
2k views

Using autocorrelation to find commonly occurring signal fragments

I have a sensor which is capturing accelerometer data as a person walks. What I'm interested in extracting is each signal fragment when a step is taken. The Z-axis is what is used since only one axis ...
54
votes
11answers
12k views

Resources for learning Markov chain and hidden Markov models

I am looking for resources (tutorials, textbooks, webcast, etc) to learn about Markov Chain and HMMs. My background is as a biologist, and I'm currently involved in a bioinformatics-related project. ...
21
votes
2answers
11k views

Markov Process about only depending on previous state

I would just like someone to confirm my understanding or if I'm missing something. The definition of a markov process says the next step depends on the current state only and no past states. So, let'...
11
votes
7answers
3k views

How should one approch Project Euler problem 213 (“Flea Circus”)?

I would like to solve Project Euler 213 but don't know where to start because I'm a layperson in the field of Statistics, notice that an accurate answer is required so the Monte Carlo method won't ...
5
votes
1answer
328 views

Lumping in Markov process with absorbing states

I have a four-state, discrete time Markov process with time-dependent transition matrices such that after a given time T the matrices become constant. The idea is people in a program leaving the ...
5
votes
2answers
272 views

Comparing noisy data sequences to estimate the likelihood of them being produced by different instances of an identical Markov process

(Prompted to some extent by the answers already given by Shane and Srikant, I've rewritten this to try to clarify what I'm getting at, if only to myself.) Suppose we have several similar systems, ...