# Questions tagged [markov-process]

A stochastic process with the property that the future is conditionally independent of the past, given the present.

953 questions
Filter by
Sorted by
Tagged with
125 views

### In a hidden Markov model, how do all observations and one state give you all states?

$Y^n$ are the observations of our HMM, where $Y_i=a_i$ is a single observation, where $a_i \epsilon \{0,1\}$. For example, $Y^n = k^n$ where $k^n=\{0,1,1,0\}$ $X^n$ are the actual states of our HMM, ...
260 views

### Inferring transition matrices in continuous time Markov processes

Suppose I have a process X1, for which I do not have a generator matrix, only a transition (probability) matrix P1 for some time interval T, e.g. T=100. Suppose I have another process X2, such that X2 ...
478 views

### Probability of several simultaneous events in Markov chains

I will state the problem in a simplified manner first. Suppose I have two objects, that can be in one of two states. We count the number times when the object is in the second state. The ...
156 views

### Confusion regarding CRF

I was reading this paper related to conditional random fields http://www.inference.phy.cam.ac.uk/hmw26/papers/crf_intro.pdf. However, I have some confusion related to the section CRF probability as ...
172 views

### Confusion regarding random walk model

I was referring to this book where it is given that If we assume equally spaced nodes $i$ for $i=1,...,n$. The first order random walk is constructed using independent increments  \Delta{x_i} \...
6k views

### What are the potential functions of the cliques in Markov random field?

I have been trying to understand the representation of the joint probability density of Markov random fields in the form of factors of the potential functions. I am finding it difficult to grasp the ...
3k views

### Test for markov-property in a time-series

Given an (observed) time-series $X_t$ with $X_t\in\{1,...,n\}$, is there a statistical test for testing the null-hypothesis that $P(X_t|X_{t-1},X_{t-2},...,X_1)=P(X_t|X_{t-1})$ (i.e. the markov-...
357 views

### Do hidden Markov models contain Markov chains?

Is it correct to say that the Hidden State Sequence in a Hidden Markov Model is a Markov Chain? Thanks
659 views

### Finding the generator matrix for a Markov jump process

Vehicles in a certain country are required to be assessed every year for road-worthiness. At one vehicle assessment center, drivers wait for an average of 15 minutes before the road-worthiness ...
42k views

### Calculate Transition Matrix (Markov) in R

Is there a way in R (a built-in function) to calculate the transition matrix for a Markov Chain from a set of observations? For example, taking a data set like the following and calculate the first ...
1k views

### Markov chain convergence, total variation and KL divergence

I have a few related questions regarding the convergence of continuous-state Markov chains. The theorems that I found claim that Markov chains converge in total variation if they are $\phi$-...
247 views

### Variance stabilization “rule” for MCMC jumps…anyone?

I have an implementation of an MCMC algorithm (Metropolis-Hastings and Adaptive Metropolis-Hastings) that I want to modify to suit my needs (it's pyMC, if anyone is interested on the details). My ...
2k views

### What is the probability of rolling all faces of a die after n number of rolls

It is fairly easy to figure out what is the average number of rolls it would take to roll all faces of a die [$1 + 6/4 + 6/4 + 6/3 + 6/2 + 6/1 = 14.7$], but that got me thinking of a seemingly more ...
3k views

### Difficulty in understanding Hidden Markov Model for syntax parsing using Viterbi algorithm

I intend to apply Kevin Murphy's Hidden markov model (HMM) toolbox. I have a set of production rules(arbitrary) $A_0 \to AB [p=1]$, $A\to aC [p=1]$, $B\to bbC [p=0.5]$, $B\to b [p=0.5]$ where $A_0$ is ...
7k views

### Problem in discrete valued time series forecasting

I have a temporally ordered discrete valued data. The only possible states for the data are: {1,2,3,4,5,6}. So the series is something like {1,2,3,5,6,4,3,5,2,......} I want to forecast the next value ...
4k views

### Markov models with conditional transition probabilities

First, let me acknowledge up front that I'm not as well versed in statistics and mathematics as I'd like to be. Some might say have just enough knowledge to be dangerous. :D I apologize if I'm not ...
760 views

### How to sample natural numbers, such that the sum is equal to a constant?

Say I have $N$ items that are partitioned / clustered and I want to randomly repartition these items, such that the distribution of sizes of the clusters is 'similar' to those that I already have. I'm ...
502 views

### MRF MAP inference for non-submodular pairwise terms

I have a multilabel MRF MAP inference problem (a labeling problem). The graph has relatively few nodes, about a thousand or so. The pairwise term is (very) not submodular (it does not satisfy the ...
6k views

### Expected number of coin tosses to get N consecutive, given M consecutive

Interviewstreet had their second CodeSprint in January that included the question below. The programmatic answer is posted but doesn't include a statistical explanation. (You can see the original ...
4k views

### Markov chain long run probabilities

So I want to find the long run probability of getting two heads in a row when flipping a fair coin many times. I know this answer should be 1/6 (expected number of flips to get 2H in a row is 6) but I'...
151 views

### Estimating an unknown restricted Markov Chain from partial measurements

There is an Markov chain $M$ defined on states $1, ..., N$ with the special property that it only has transitions $p_i$ from $i$ to $i + 1$ , $q_{i + 1}$ from $i + 1$ to $i$, and $r_i = 1 - p_i - q_i$ ...
2k views

### Are two empirically estimated Markov chains statistically different?

I am constructing Markov chains (with 100 to 200 states) and inferring transition probabilities empirically by simply counting how many times I saw each transition in my raw data (about 20k to 60k ...
397 views

### Stationary distribution on a partition of the state space?

I would like to use the following type of model and am wondering if someone could let me know if this technique has been studied before (I assume it has) and where to learn more about it? The idea is ...
504 views

### Modeling null transitions in the Hidden Markov Model for use with the Viterbi algorithm

I've implemented the classic HMM model from Rabiner's tutorial for gesture recognition and it has worked well. Now, I'm trying to implement the HMM Threshold Model which calls for an HMM with null ...
106 views

### Statistically back-calculating: Markov Chain?

I would like to calculate the value of bacteria on 4 surfaces $i=\{1..4\}$. A person touches some of those 4 surfaces at random and a count is made on their finger after each surface contact ($x_i$). ...
6k views

### Where can I find C++ code samples for HMM (hidden Markov model) as it relates to gesture recognition? [closed]

Any help would be greatly appreciated.
33k views

### What are the differences between hidden Markov models and neural networks?

I'm just getting my feet wet in statistics so I'm sorry if this question does not make sense. I have used Markov models to predict hidden states (unfair casinos, dice rolls, etc.) and neural networks ...
109 views

### having trouble applying hidden markov models to my game [duplicate]

Possible Duplicate: having trouble applying hidden markov/machine learning models Happy New Year! I’m having a problem applying hidden Markov models to a game I’m building to learn about ...
765 views

### Hidden markov models: output observations defined by a (non-hidden) markov model?

Let me explain what my goal is: I would like to define a hidden markov model with two hidden states and say, five possible observations. As I understand (I'm quite new to HMMs), in each state HMM will ...
11k views

### Examples of hidden Markov models problems?

I read quite a bit of hidden Markov models and was able to code a pretty basic version of it myself. But there are two main ways I seem to learn. One is to read and implement it into code (which is ...
248 views

### Summing values of state transitions accumulated in an absorbing Markov chain

I am trying to simulate a process as an absorbing Markov chain model, but I haven't been able to find the scenario that I am interested in looking at in the typical discussions of Markov chains online....
418 views

### Confidence intervals for difference in time series

I have a stochastic model used to simulate time series of some process. I am interested in the effect of changing one parameter to a specific value and want to show the difference between the time ...
1k views

### Random matrices with constraints on row and column length

I need to generate random non-square matrices with $R$ rows and $C$ columns, elements randomly distributed with mean = 0, and constrained such that the length (L2 norm) of each row is $1$ and the ...
387 views

### Figuring out probabilities with Hidden Markov Models

I'm really new to statistics so sorry in advance if this question does not make sense. Background: I'm trying to learn about hidden Markov models and they seem interesting but I was wondering about ...
467 views

### What methods of statistical analysis can be used for time series data?

I have done many 1-sample T-tests before, but I can't figure out if I am able to use one in this situation. In our experiment, we took 12 individual insects and placed them in a chamber where they ...
7k views

### Hidden Markov models and expectation maximization algorithm

Can somebody clarify how hidden Markov models are related to expectation maximization? I have gone through many links but couldn't come up with a clear view. Thanks!
2k views

### Maximum likelihood estimation procedures for state-space linear models

State-space models are represented by a state equation and an observation equation (or system of equations to be more precise). These equations are parametarized by components including a transition ...
15k views

### Estimating Markov chain probabilities

What would be the common way of estimating MC transition matrix given the timeseries? Is there R function for doing that?
400 views

### CDF of first passage time for Poisson dam/shot noise with constant amplitude jumps and exponential decay

I'm looking for a CDF (either exact or approximation) of the first passage time, $T$, of a storage process $\{X(t); 0 \leq t < \infty \},$ with constant (unit) input jumps occurring in a Poisson ...
228 views

### Sampler method to choose in Monte Carlo Markov chain estimation

When estimating the posterior via MCMC, are there guidelines on the best sampling method to use depending on the nature of the model? There are a variety of forms of MCMC - the Gibbs sampler, the ...
976 views

### How can I generate correlated timeseries made up of 0s and 1s?

I want to generate series of 0s and 1s that exhibit some clustering. By this I mean that 1s and 0s should occur together. So I envisage series of 0s and 1s that will exhibit similar clustering of ...
2k views

### Do we have a problem of “pity upvotes”?

I know, this may sound like it is off-topic, but hear me out. At Stack Overflow and here we get votes on posts, this is all stored in a tabular form. E.g.: post id voter id vote type ...
1k views

### Steady state probabilities for a continuous-time Markov chain

I have a finite state and time-homogeneous continuous-time Markov chain (CTMC) which is not irreducible. Will steady state probabilities exist for this CTMC? How to prove this?
790 views

### Discrepancy measures for transition matrices

I'm doing some work on modelling transition matrices, and for this I need a measure of discrepancy or lack of fit: that is, if I have a matrix $T$ and a target matrix $T_0$, I want to be able to ...
477 views

### Can we get confidence intervals for entries in stationary vector for an absorbing, time-independent Markov chain?

I have a finite-state, time-independent Markov chain with two absorbing states which models educational outcomes (the absorbing states are completing and not completing). The transition probabilities ...
709 views

### First passage time distribution in a irreducible transient discrete-time Markov chain (DTMC)

In a Markov chain, a state $j$ is transient if $f_{jj}<1$ ($f_{jj}$ is probability of ever visiting state $j$ starting from state $j$ ). Suppose, I have an irreducible transient DTMC (means all ...
348 views

### Uncertainty of conditional probability evaluated from sample

I'm doing some some analysis of an arbitrary string of text, modelling it as a Markov chain where the state is simply the value of the previous character. Call the current character ...
2k views

### Using autocorrelation to find commonly occurring signal fragments

I have a sensor which is capturing accelerometer data as a person walks. What I'm interested in extracting is each signal fragment when a step is taken. The Z-axis is what is used since only one axis ...