A stochastic process with the property that the future is conditionally independent of the past, given the present.

learn more… | top users | synonyms

-1
votes
0answers
12 views

markov proof for poisson process and question 2 [on hold]

I want answers for those 3 questions as soon as possible.I can't understad those stuffs.
0
votes
0answers
12 views

Application of Lévy–Khinchine formula

How can we express the characteristic functions of Wiener and Poisson processes by using the Lévy–Khinchine formula? I don't know how to find the characteristic functions of particular Levy ...
0
votes
0answers
23 views

Have a data set for 3 consecutive days. What are my options?

Let's say I have a set regarding the transportation methods(Eg: car, bus, train) used for three consecutive days b y $n$ number of people. For simplicity let us assume that everyone use only one type ...
0
votes
0answers
18 views

Modelling remaing time of a process

I have process which has different states. It looks something like this: In some cases the required tools for the assembly need to be fetched (same goes for the supplies for packaging). Typical ...
0
votes
1answer
38 views

Estimating Standard Errors for Markov Transition Probability with Multiple Observations (in R)

I was trying to estimate a Markov transition table from paired transition data, which look something like this: ...
0
votes
1answer
28 views

Markov Chains : Can anything be said about what happens in between two transition?

In time homogeneous discrete Markov chains we take a set period for a single transition. In examples we see sometimes depending on the examples the transition period being a a month a week etc. I'm ...
0
votes
1answer
23 views

Modelling a probability distribution on different feature sets

I have a binary classification problem, and I use method A and method B to extract features, F1 and F2, for this problem from dataset X. Now, I train two models, y1 and y2, separately on the two ...
0
votes
0answers
8 views

sample generative model from a chain/tree

I have a tree of states and I would like to sample from this tree based on pure birth process; however, I don't know how exactly I can do this; so far I have done this; I simplified my problem; the ...
0
votes
0answers
25 views

Probabilities in a Markov Model

I am reading a paper on Markov Models and I am trying to figure out how to compute the probabilities for the $\alpha$-pass. I am given an $N\times N$ matrix $A$, that has the probabilities of ...
1
vote
1answer
32 views

MDP value iteration

In Markov decision processes, what is the guarantee that value iteration chooses the same policy action from a given state for every iteration? I am referring to the slides given by AWM at ...
0
votes
0answers
26 views

Continuous time markov chains, is this step by step example correct

Following my initial question (stats.stackexchange.com/questions/108874/continuous-time-markov-chains-simple-explanation) and some further reading on CTMC ([1], and [2] and also ...
0
votes
0answers
31 views

Continuous time markov chains simple explanation

I am looking into continuous time markov chains which I think are capable of answering questions like "what is the probability that after time $t$ (where $t$ is in discrete time steps in this case) a ...
0
votes
0answers
21 views

Maxent Markov models in R

Is there a package that implements Maxent Markov Models in R? (http://en.wikipedia.org/wiki/Maximum-entropy_Markov_model). I understand that package crf implemets conditional random fields which are ...
2
votes
1answer
25 views

Is it possible to determine the probability of NOT reaching a state in a FSA ?

Suppose I have a Finite State Automata with various states and probabilities of state transitions. Does mathematics exist to determine the probability of NOT reaching a state in the FSA given some n ...
0
votes
1answer
54 views

Transition rates in continuous time markov chain

A house has 2 rooms of similar sizes with identical air conditioners equipped with thermostats which turn on and off as needed to maintain the temperature in each room to a desired level of 22 ...
0
votes
0answers
21 views

Transition matrix in left-right hidden semi-Markov model

i'm developing a hidden semi-Markov model left-right . In a left-right model a sequence of $M$ states starts in state 1 and ends in state M, with no repetition of states. Since the model is ...
4
votes
0answers
22 views

How Should I Find Statistically Significant Differences Between Two Markov Models?

Suppose I have N Markov Models of M states representing the behaviour patterns of 2 different groups (note: fully observable models, no hidden states), and have stored each model as a matrix of ...
3
votes
0answers
31 views

Convergence time of a Markov chain

We know that a regular Markov chains converges to a unique matrix. The convergence time maybe finite or infinite. My interest is in the case where the convergence time is finite. How can we accurately ...
0
votes
0answers
45 views

How would you validate a random walk model?

I have used a random walk model and Gibbs sampling (more specifically RJAGS) in order to obtain posterior of the state given the observations. In this case the state is the true proportion of the ...
1
vote
0answers
49 views

Concavity of log likelihood for hidden markov models

Could you give me a good link where the concept of concavity of the log likelihood related to hidden markov model EM algorithm is clarified? Thank you in advance.
0
votes
0answers
34 views

HIdden Markov model training Baum Welch, concavity log likelihood

Hi i'm developing an hidden markov model algorithm training with multiple sequences. The recognition rate is good but i have doubts about the shape of the curve of log likelihood obtained from the ...
1
vote
1answer
88 views

Difference between Bayesian Networks & Markov Process

I am new to stats and didn't have the chance to take the course at university. However I do need to know the difference between Bayesian Networks & Markov Processes. I found good material for ...
0
votes
0answers
38 views

Continuous time markov chain backward/forward equations

Using Kolmogorov's forward and backward equations, show that $p_{11}(t) + p_{21}(t) + p_{31}(t) = 1$ and $p_{21}(t) = p_{31}(t)$ where $p_{ij}(t) = P(X(t) = j | X(0) = i)$. My attempt: I can show ...
0
votes
0answers
46 views

Continuous Time Markov Chain Transition Rates

A hospital has two physicians on call, Dr Dawson and Dr Baick. Dr Dawson is available to answer patients' calls for time periods that are exponentially distributed with mean 2 hours. Between ...
0
votes
0answers
36 views

Distance between a transition matrix and an instance

I am trying to put a number to the distance of a sequence and how close it is to the original training corpus. From the original training data, I got a markov transition matrix (TM). So from the ...
0
votes
1answer
75 views

Discrete time Markov Chain - Long-term frequency

Let's say I have the following scenario: A mouse is put into a maze that's constructed as below: There are 9 rooms with connections between the rooms as indicated with a "gap" in the ...
0
votes
0answers
79 views

Difference between fuzzy graphical network , Markov model and Bayesian network

Referring to this answer Difference between Bayesian network and neural network and causal inference, I have come across other graphical models (1) Fuzzy Cognitive Map and (2) Neuro-Fuzzy (3) Fuzzy ...
0
votes
0answers
16 views

How to validate classifier (built by using MLN method)?

I have developed a method (let's call it Method X) that has a classifier function. The classifier function was built by using MLN (Markov logic network). I need to ...
1
vote
1answer
48 views

Maximum likelihood estimation (MLE) for Markov Chain initial distribution?

I am working on using MLE to estimate a Markov Chain, I have successfully estimated the transition matrix $A$, using the method provided in ...
0
votes
1answer
49 views

Topic and subject classification

I have a set of documents that are OCR-ed and represented as a text file. I want to find out what are the documents that are talking about the same subject and maybe about the same person. I started ...
1
vote
0answers
19 views

Markov Switching and Hidden Markov Models

Are the two interchangeable terms? I have been reading about markov-switching models and am struggling to see the difference with HMM models.
0
votes
0answers
24 views

What should be done to deal with missing observations ( or outlier observation ) for Viterbi?

I want to use Viterbi algorithm, to decode an HMM sequence, but very few observations are missing in some of the steps or outliers. The hidden states in these steps are assumed to be the same as ...
1
vote
0answers
17 views

Markov-Switching model of distribution

I was wondering if anyone knows of any work concerning markov-switching models and more precisely that changes the distribution of the error terms? I've found some literature for conditional mean, ...
0
votes
0answers
8 views

Random walks in Markov-Switching states

Does anyone know of any literature which discusses the idea of random walk processes in states of a Markov-Switching model? I am struggling to find anything which discusses the theory/fitting behind ...
1
vote
0answers
90 views

How to forecast a Markov Switching Model

I have the following Markov Switching Model. Transition Matrix: $$ \left[\begin{matrix} 0.85387 & 0.91973\\0.14613 & 0.080265 \end{matrix}\right] $$ With Regime 1: Intercept: 0.00839 ...
0
votes
0answers
14 views

Determining the state of a conditional system

I have the following situation: There are two systems, $s_A$ and $s_B$. For $s_A$ there are two states: $s_A = a,b$. For $s_B$ there are three states: $s_B = 1,2,3$. Now, what I want to do is measure ...
2
votes
0answers
72 views

How to find a conditional probability using copula-based Markov process?

I have a monthly time series of a water quality parameter. I used copula-based Markov process of C(Y(t), Y(t-1) and I forecasted the mean behavior of Yt by following equation: Now, I need to find ...
2
votes
1answer
79 views

What are $\rho$-, $\beta$-, and $\alpha$-mixing conditions?

I have seen properties named $\rho$-, $\beta$-, and $\alpha$-mixing conditions in papers related to Copulas and Markov processes like this one: In this paper, we identify conditions on $C$ that ...
1
vote
1answer
57 views

Difference between iid data and non-iid data for a simple regression problem

I am trying to understand the difference between iid and non-iid data. Let's consider a given time series, and say it's reasonable to assume that at each time point the random variable $X_t$ depends ...
2
votes
0answers
49 views

When is the autocorrelation function of a stationary process decreasing/nonincreasing? Markovian?

When is the autocorrelation function of a stationary process strictly decreasing or nonincreasing? Can being Markovian make it true? When is the autocorrelation function of a stationary process ...
2
votes
1answer
106 views

Comparing the distributions of two processes, one of which is constrained by zero

I have two continuous stochastic Markov processes: the concentration readout of two proteins in a cell over time. These are shown in this figure, where the blue line is the unbounded protein, and all ...
0
votes
1answer
25 views

Markov model for time series, going back n periods?

I realise there are a lot of questions about markov here, but as we say in Dutch, I couldn't see the threes through the forest. I have a sequence of intervals between subsequent notes (pitches). ...
2
votes
0answers
54 views

Markov chain and process

This is not homework. Just practicing for an upcoming exam. Question is taken from a web pdf : http://www.dartmouth.edu/~chance/teaching_aids/books_articles/probability_book/Chapter11.pdf Consider ...
0
votes
1answer
26 views

How can a probability distribution P not factorize over a graph H when P satisfies the independencies implied by H

How can a probability distribution P not factorize over a graph H when P satisfies the all the global independencies implied by H? Here's an example: Let $X_1, \dots X_4$ be 4 random variables that ...
0
votes
1answer
12 views

Way to avoid predefining actions and states in reinforcement learning

Recently, I have heard of the concept of the reinforcement learning and I have got interested in it. So I decided to start a project that uses this kind of machine learning for constructing algorithms ...
2
votes
1answer
32 views

Mixture of normals, dependent on past

I have the following probability model: $(X_k|\text{PastHistory}_{k-1}, \theta_0,\theta_1,\theta_2) \sim (\pi\cdot N(\theta_1+\theta_0\cdot X_{k-1},1)+(1-\pi)\cdot N(\theta_2+\theta_0\cdot ...
0
votes
1answer
40 views

What Bayesian model should I use for posterior room identification?

I have n rooms (which can be considered as states) and a sensor on my robot which gives me a probability array of what room it is in (this array is of size n and its sum is 1). At every timestamp (the ...
0
votes
0answers
53 views

Discrete Time Markov Chain - Inventory

Let $D_n$ be the demand for an item at a store on day $n$. Suppose that $D_n$ is a sequence of independent and identically distributed (i.i.d.) random variables with probability mass function: $p_k = ...
3
votes
0answers
59 views

Continuous time Markov chain forward equation

This is a homework question and I need suggestion how to approach it. We have given the transitions $\ i\rightarrow i+1$ with rate $\lambda(i)$ where $\ i \ge 1$ $\ i\rightarrow i-1$ with rate ...
1
vote
0answers
153 views

Easy to follow tutorial on using Markov Random Fields for classifying pixels in gray-scale images

I am trying to learn how to use Markov Random Fields for classifying pixels in an image. Could someone please direct me to a simple tutorial demonstrating how this is done. The tutorial needs to ...