Questions tagged [markov-process]

A stochastic process with the property that the future is conditionally independent of the past, given the present.

956 questions
Filter by
Sorted by
Tagged with
96 views

Markov random field and iterated condition mode

I have spent a lot of time studying MRF (applied to images) but still can't grasp the idea. Could you please clarify these ideas: What is the clique potential? What is a clique in image, and do they ...
401 views

Stationary matrix given a transition matrix

I am given the following transition matrix $$P= \pmatrix{ 1-\alpha & \alpha \\ \beta & 1-\beta}, \ \alpha,\beta \in (0,1)$$ with the states $S=\{1,2\}$. I want to determine the stationary ...
438 views

229 views

Sampler method to choose in Monte Carlo Markov chain estimation

When estimating the posterior via MCMC, are there guidelines on the best sampling method to use depending on the nature of the model? There are a variety of forms of MCMC - the Gibbs sampler, the ...
3k views

Test for markov-property in a time-series

Given an (observed) time-series $X_t$ with $X_t\in\{1,...,n\}$, is there a statistical test for testing the null-hypothesis that $P(X_t|X_{t-1},X_{t-2},...,X_1)=P(X_t|X_{t-1})$ (i.e. the markov-...
664 views

Finding the generator matrix for a Markov jump process

Vehicles in a certain country are required to be assessed every year for road-worthiness. At one vehicle assessment center, drivers wait for an average of 15 minutes before the road-worthiness ...
1k views

Markov chain convergence, total variation and KL divergence

I have a few related questions regarding the convergence of continuous-state Markov chains. The theorems that I found claim that Markov chains converge in total variation if they are $\phi$-...
2k views

Are two empirically estimated Markov chains statistically different?

I am constructing Markov chains (with 100 to 200 states) and inferring transition probabilities empirically by simply counting how many times I saw each transition in my raw data (about 20k to 60k ...
3k views

Difficulty in understanding Hidden Markov Model for syntax parsing using Viterbi algorithm

I intend to apply Kevin Murphy's Hidden markov model (HMM) toolbox. I have a set of production rules(arbitrary) $A_0 \to AB [p=1]$, $A\to aC [p=1]$, $B\to bbC [p=0.5]$, $B\to b [p=0.5]$ where $A_0$ is ...
2k views

What is the probability of rolling all faces of a die after n number of rolls

It is fairly easy to figure out what is the average number of rolls it would take to roll all faces of a die [$1 + 6/4 + 6/4 + 6/3 + 6/2 + 6/1 = 14.7$], but that got me thinking of a seemingly more ...
7k views

Problem in discrete valued time series forecasting

I have a temporally ordered discrete valued data. The only possible states for the data are: {1,2,3,4,5,6}. So the series is something like {1,2,3,5,6,4,3,5,2,......} I want to forecast the next value ...
504 views

Modeling null transitions in the Hidden Markov Model for use with the Viterbi algorithm

I've implemented the classic HMM model from Rabiner's tutorial for gesture recognition and it has worked well. Now, I'm trying to implement the HMM Threshold Model which calls for an HMM with null ...
764 views

How to sample natural numbers, such that the sum is equal to a constant?

Say I have $N$ items that are partitioned / clustered and I want to randomly repartition these items, such that the distribution of sizes of the clusters is 'similar' to those that I already have. I'm ...
6k views

Expected number of coin tosses to get N consecutive, given M consecutive

Interviewstreet had their second CodeSprint in January that included the question below. The programmatic answer is posted but doesn't include a statistical explanation. (You can see the original ...
4k views

Markov chain long run probabilities

So I want to find the long run probability of getting two heads in a row when flipping a fair coin many times. I know this answer should be 1/6 (expected number of flips to get 2H in a row is 6) but I'...
398 views

Stationary distribution on a partition of the state space?

I would like to use the following type of model and am wondering if someone could let me know if this technique has been studied before (I assume it has) and where to learn more about it? The idea is ...
106 views

Statistically back-calculating: Markov Chain?

I would like to calculate the value of bacteria on 4 surfaces $i=\{1..4\}$. A person touches some of those 4 surfaces at random and a count is made on their finger after each surface contact ($x_i$). ...
109 views

having trouble applying hidden markov models to my game [duplicate]

Possible Duplicate: having trouble applying hidden markov/machine learning models Happy New Year! I’m having a problem applying hidden Markov models to a game I’m building to learn about ...
1k views

Steady state probabilities for a continuous-time Markov chain

I have a finite state and time-homogeneous continuous-time Markov chain (CTMC) which is not irreducible. Will steady state probabilities exist for this CTMC? How to prove this?
350 views

Lumping in Markov process with absorbing states

I have a four-state, discrete time Markov process with time-dependent transition matrices such that after a given time T the matrices become constant. The idea is people in a program leaving the ...
248 views

Summing values of state transitions accumulated in an absorbing Markov chain

I am trying to simulate a process as an absorbing Markov chain model, but I haven't been able to find the scenario that I am interested in looking at in the typical discussions of Markov chains online....
467 views

What methods of statistical analysis can be used for time series data?

I have done many 1-sample T-tests before, but I can't figure out if I am able to use one in this situation. In our experiment, we took 12 individual insects and placed them in a chamber where they ...
2k views

Maximum likelihood estimation procedures for state-space linear models

State-space models are represented by a state equation and an observation equation (or system of equations to be more precise). These equations are parametarized by components including a transition ...