Questions tagged [markov-process]

A stochastic process with the property that the future is conditionally independent of the past, given the present.

Filter by
Sorted by
Tagged with
2
votes
0answers
125 views

In a hidden Markov model, how do all observations and one state give you all states?

$Y^n$ are the observations of our HMM, where $Y_i=a_i$ is a single observation, where $a_i \epsilon \{0,1\}$. For example, $Y^n = k^n$ where $k^n=\{0,1,1,0\}$ $X^n$ are the actual states of our HMM, ...
2
votes
0answers
260 views

Inferring transition matrices in continuous time Markov processes

Suppose I have a process X1, for which I do not have a generator matrix, only a transition (probability) matrix P1 for some time interval T, e.g. T=100. Suppose I have another process X2, such that X2 ...
2
votes
0answers
478 views

Probability of several simultaneous events in Markov chains

I will state the problem in a simplified manner first. Suppose I have two objects, that can be in one of two states. We count the number times when the object is in the second state. The ...
2
votes
0answers
156 views

Confusion regarding CRF

I was reading this paper related to conditional random fields http://www.inference.phy.cam.ac.uk/hmw26/papers/crf_intro.pdf. However, I have some confusion related to the section CRF probability as ...
5
votes
2answers
172 views

Confusion regarding random walk model

I was referring to this book where it is given that If we assume equally spaced nodes $i$ for $i=1,...,n$. The first order random walk is constructed using independent increments $$ \Delta{x_i} \...
6
votes
1answer
6k views

What are the potential functions of the cliques in Markov random field?

I have been trying to understand the representation of the joint probability density of Markov random fields in the form of factors of the potential functions. I am finding it difficult to grasp the ...
10
votes
1answer
3k views

Test for markov-property in a time-series

Given an (observed) time-series $X_t$ with $X_t\in\{1,...,n\}$, is there a statistical test for testing the null-hypothesis that $P(X_t|X_{t-1},X_{t-2},...,X_1)=P(X_t|X_{t-1})$ (i.e. the markov-...
0
votes
2answers
357 views

Do hidden Markov models contain Markov chains?

Is it correct to say that the Hidden State Sequence in a Hidden Markov Model is a Markov Chain? Thanks
2
votes
1answer
659 views

Finding the generator matrix for a Markov jump process

Vehicles in a certain country are required to be assessed every year for road-worthiness. At one vehicle assessment center, drivers wait for an average of 15 minutes before the road-worthiness ...
29
votes
2answers
42k views

Calculate Transition Matrix (Markov) in R

Is there a way in R (a built-in function) to calculate the transition matrix for a Markov Chain from a set of observations? For example, taking a data set like the following and calculate the first ...
5
votes
1answer
1k views

Markov chain convergence, total variation and KL divergence

I have a few related questions regarding the convergence of continuous-state Markov chains. The theorems that I found claim that Markov chains converge in total variation if they are $\phi$-...
6
votes
1answer
247 views

Variance stabilization “rule” for MCMC jumps…anyone?

I have an implementation of an MCMC algorithm (Metropolis-Hastings and Adaptive Metropolis-Hastings) that I want to modify to suit my needs (it's pyMC, if anyone is interested on the details). My ...
5
votes
1answer
2k views

What is the probability of rolling all faces of a die after n number of rolls

It is fairly easy to figure out what is the average number of rolls it would take to roll all faces of a die [$1 + 6/4 + 6/4 + 6/3 + 6/2 + 6/1 = 14.7$], but that got me thinking of a seemingly more ...
6
votes
1answer
3k views

Difficulty in understanding Hidden Markov Model for syntax parsing using Viterbi algorithm

I intend to apply Kevin Murphy's Hidden markov model (HMM) toolbox. I have a set of production rules(arbitrary) $A_0 \to AB [p=1]$, $A\to aC [p=1]$, $B\to bbC [p=0.5]$, $B\to b [p=0.5]$ where $A_0$ is ...
3
votes
2answers
7k views

Problem in discrete valued time series forecasting

I have a temporally ordered discrete valued data. The only possible states for the data are: {1,2,3,4,5,6}. So the series is something like {1,2,3,5,6,4,3,5,2,......} I want to forecast the next value ...
9
votes
3answers
4k views

Markov models with conditional transition probabilities

First, let me acknowledge up front that I'm not as well versed in statistics and mathematics as I'd like to be. Some might say have just enough knowledge to be dangerous. :D I apologize if I'm not ...
4
votes
3answers
760 views

How to sample natural numbers, such that the sum is equal to a constant?

Say I have $N$ items that are partitioned / clustered and I want to randomly repartition these items, such that the distribution of sizes of the clusters is 'similar' to those that I already have. I'm ...
5
votes
2answers
502 views

MRF MAP inference for non-submodular pairwise terms

I have a multilabel MRF MAP inference problem (a labeling problem). The graph has relatively few nodes, about a thousand or so. The pairwise term is (very) not submodular (it does not satisfy the ...
10
votes
3answers
6k views

Expected number of coin tosses to get N consecutive, given M consecutive

Interviewstreet had their second CodeSprint in January that included the question below. The programmatic answer is posted but doesn't include a statistical explanation. (You can see the original ...
2
votes
2answers
4k views

Markov chain long run probabilities

So I want to find the long run probability of getting two heads in a row when flipping a fair coin many times. I know this answer should be 1/6 (expected number of flips to get 2H in a row is 6) but I'...
4
votes
1answer
151 views

Estimating an unknown restricted Markov Chain from partial measurements

There is an Markov chain $M$ defined on states $1, ..., N$ with the special property that it only has transitions $p_i$ from $i$ to $i + 1$ , $q_{i + 1}$ from $i + 1$ to $i$, and $r_i = 1 - p_i - q_i$ ...
5
votes
2answers
2k views

Are two empirically estimated Markov chains statistically different?

I am constructing Markov chains (with 100 to 200 states) and inferring transition probabilities empirically by simply counting how many times I saw each transition in my raw data (about 20k to 60k ...
2
votes
1answer
397 views

Stationary distribution on a partition of the state space?

I would like to use the following type of model and am wondering if someone could let me know if this technique has been studied before (I assume it has) and where to learn more about it? The idea is ...
4
votes
1answer
504 views

Modeling null transitions in the Hidden Markov Model for use with the Viterbi algorithm

I've implemented the classic HMM model from Rabiner's tutorial for gesture recognition and it has worked well. Now, I'm trying to implement the HMM Threshold Model which calls for an HMM with null ...
1
vote
1answer
106 views

Statistically back-calculating: Markov Chain?

I would like to calculate the value of bacteria on 4 surfaces $i=\{1..4\}$. A person touches some of those 4 surfaces at random and a count is made on their finger after each surface contact ($x_i$). ...
0
votes
1answer
6k views

Where can I find C++ code samples for HMM (hidden Markov model) as it relates to gesture recognition? [closed]

Any help would be greatly appreciated.
40
votes
3answers
33k views

What are the differences between hidden Markov models and neural networks?

I'm just getting my feet wet in statistics so I'm sorry if this question does not make sense. I have used Markov models to predict hidden states (unfair casinos, dice rolls, etc.) and neural networks ...
0
votes
0answers
109 views

having trouble applying hidden markov models to my game [duplicate]

Possible Duplicate: having trouble applying hidden markov/machine learning models Happy New Year! I’m having a problem applying hidden Markov models to a game I’m building to learn about ...
3
votes
2answers
765 views

Hidden markov models: output observations defined by a (non-hidden) markov model?

Let me explain what my goal is: I would like to define a hidden markov model with two hidden states and say, five possible observations. As I understand (I'm quite new to HMMs), in each state HMM will ...
21
votes
6answers
11k views

Examples of hidden Markov models problems?

I read quite a bit of hidden Markov models and was able to code a pretty basic version of it myself. But there are two main ways I seem to learn. One is to read and implement it into code (which is ...
3
votes
0answers
248 views

Summing values of state transitions accumulated in an absorbing Markov chain

I am trying to simulate a process as an absorbing Markov chain model, but I haven't been able to find the scenario that I am interested in looking at in the typical discussions of Markov chains online....
11
votes
1answer
418 views

Confidence intervals for difference in time series

I have a stochastic model used to simulate time series of some process. I am interested in the effect of changing one parameter to a specific value and want to show the difference between the time ...
24
votes
1answer
1k views

Random matrices with constraints on row and column length

I need to generate random non-square matrices with $R$ rows and $C$ columns, elements randomly distributed with mean = 0, and constrained such that the length (L2 norm) of each row is $1$ and the ...
3
votes
1answer
387 views

Figuring out probabilities with Hidden Markov Models

I'm really new to statistics so sorry in advance if this question does not make sense. Background: I'm trying to learn about hidden Markov models and they seem interesting but I was wondering about ...
1
vote
2answers
467 views

What methods of statistical analysis can be used for time series data?

I have done many 1-sample T-tests before, but I can't figure out if I am able to use one in this situation. In our experiment, we took 12 individual insects and placed them in a chamber where they ...
10
votes
3answers
7k views

Hidden Markov models and expectation maximization algorithm

Can somebody clarify how hidden Markov models are related to expectation maximization? I have gone through many links but couldn't come up with a clear view. Thanks!
5
votes
1answer
2k views

Maximum likelihood estimation procedures for state-space linear models

State-space models are represented by a state equation and an observation equation (or system of equations to be more precise). These equations are parametarized by components including a transition ...
12
votes
3answers
15k views

Estimating Markov chain probabilities

What would be the common way of estimating MC transition matrix given the timeseries? Is there R function for doing that?
3
votes
0answers
400 views

CDF of first passage time for Poisson dam/shot noise with constant amplitude jumps and exponential decay

I'm looking for a CDF (either exact or approximation) of the first passage time, $T$, of a storage process $\{X(t); 0 \leq t < \infty \},$ with constant (unit) input jumps occurring in a Poisson ...
3
votes
1answer
228 views

Sampler method to choose in Monte Carlo Markov chain estimation

When estimating the posterior via MCMC, are there guidelines on the best sampling method to use depending on the nature of the model? There are a variety of forms of MCMC - the Gibbs sampler, the ...
5
votes
1answer
976 views

How can I generate correlated timeseries made up of 0s and 1s?

I want to generate series of 0s and 1s that exhibit some clustering. By this I mean that 1s and 0s should occur together. So I envisage series of 0s and 1s that will exhibit similar clustering of ...
50
votes
3answers
2k views

Do we have a problem of “pity upvotes”?

I know, this may sound like it is off-topic, but hear me out. At Stack Overflow and here we get votes on posts, this is all stored in a tabular form. E.g.: post id voter id vote type ...
0
votes
1answer
1k views

Steady state probabilities for a continuous-time Markov chain

I have a finite state and time-homogeneous continuous-time Markov chain (CTMC) which is not irreducible. Will steady state probabilities exist for this CTMC? How to prove this?
6
votes
2answers
790 views

Discrepancy measures for transition matrices

I'm doing some work on modelling transition matrices, and for this I need a measure of discrepancy or lack of fit: that is, if I have a matrix $T$ and a target matrix $T_0$, I want to be able to ...
6
votes
1answer
477 views

Can we get confidence intervals for entries in stationary vector for an absorbing, time-independent Markov chain?

I have a finite-state, time-independent Markov chain with two absorbing states which models educational outcomes (the absorbing states are completing and not completing). The transition probabilities ...
2
votes
1answer
709 views

First passage time distribution in a irreducible transient discrete-time Markov chain (DTMC)

In a Markov chain, a state $j$ is transient if $f_{jj}<1$ ($f_{jj}$ is probability of ever visiting state $j$ starting from state $j$ ). Suppose, I have an irreducible transient DTMC (means all ...
4
votes
2answers
348 views

Uncertainty of conditional probability evaluated from sample

I'm doing some some analysis of an arbitrary string of text, modelling it as a Markov chain where the state is simply the value of the previous character. Call the current character ...
7
votes
1answer
2k views

Using autocorrelation to find commonly occurring signal fragments

I have a sensor which is capturing accelerometer data as a person walks. What I'm interested in extracting is each signal fragment when a step is taken. The Z-axis is what is used since only one axis ...
57
votes
11answers
12k views

Resources for learning Markov chain and hidden Markov models

I am looking for resources (tutorials, textbooks, webcast, etc) to learn about Markov Chain and HMMs. My background is as a biologist, and I'm currently involved in a bioinformatics-related project. ...
22
votes
2answers
13k views

Markov Process about only depending on previous state

I would just like someone to confirm my understanding or if I'm missing something. The definition of a markov process says the next step depends on the current state only and no past states. So, let'...