A stochastic process with the property that the future is conditionally independent of the past, given the present.

learn more… | top users | synonyms

0
votes
0answers
6 views

How to validate classifier (built by using MLN method)?

I have developed a method (let's call it Method X) that has a classifier function. The classifier function was built by using MLN (Markov logic network). I need to ...
1
vote
0answers
18 views

Maximum likelihood estimation (MLE) for Markov Chain initial distribution?

I am working on using MLE to estimate a Markov Chain, I have successfully estimated the transition matrix $A$, using the method provided in ...
0
votes
1answer
22 views

Topic and subject classification

I have a set of documents that are OCR-ed and represented as a text file. I want to find out what are the documents that are talking about the same subject and maybe about the same person. I started ...
0
votes
0answers
7 views

Markov Switching and Hidden Markov Models

Are the two interchangeable terms? I have been reading about markov-switching models and am struggling to see the difference with HMM models.
0
votes
0answers
11 views

What should be done to deal with missing observations ( or outlier observation ) for Viterbi?

I want to use Viterbi algorithm, to decode an HMM sequence, but very few observations are missing in some of the steps or outliers. The hidden states in these steps are assumed to be the same as ...
1
vote
0answers
11 views

Markov-Switching model of distribution

I was wondering if anyone knows of any work concerning markov-switching models and more precisely that changes the distribution of the error terms? I've found some literature for conditional mean, ...
0
votes
0answers
4 views

Random walks in Markov-Switching states

Does anyone know of any literature which discusses the idea of random walk processes in states of a Markov-Switching model? I am struggling to find anything which discusses the theory/fitting behind ...
1
vote
0answers
31 views

How to forecast a Markov Switching Model

I have the following Markov Switching Model. Transition Matrix: $$ \left[\begin{matrix} 0.85387 & 0.91973\\0.14613 & 0.080265 \end{matrix}\right] $$ With Regime 1: Intercept: 0.00839 ...
0
votes
0answers
12 views

Determining the state of a conditional system

I have the following situation: There are two systems, $s_A$ and $s_B$. For $s_A$ there are two states: $s_A = a,b$. For $s_B$ there are three states: $s_B = 1,2,3$. Now, what I want to do is measure ...
1
vote
0answers
24 views

How to find a conditional probability using copula-based Markov process?

I have a monthly time series of a water quality parameter. I used copula-based Markov process of C(Y(t), Y(t-1) and I forecasted the mean behavior of Yt by following equation: Now, I need to find ...
0
votes
0answers
21 views

What are $\rho$-, $\beta$-, and $\alpha$-mixing conditions?

I have seen properties named $\rho$-, $\beta$-, and $\alpha$-mixing conditions in papers related to Copulas and Markov processes like this one: In this paper, we identify conditions on $C$ that ...
1
vote
1answer
31 views

Difference between iid data and non-iid data for a simple regression problem

I am trying to understand the difference between iid and non-iid data. Let's consider a given time series, and say it's reasonable to assume that at each time point the random variable $X_t$ depends ...
2
votes
0answers
35 views

When is the autocorrelation function of a stationary process decreasing/nonincreasing? Markovian?

When is the autocorrelation function of a stationary process strictly decreasing or nonincreasing? Can being Markovian make it true? When is the autocorrelation function of a stationary process ...
2
votes
1answer
96 views

Comparing the distributions of two processes, one of which is constrained by zero

I have two continuous stochastic Markov processes: the concentration readout of two proteins in a cell over time. These are shown in this figure, where the blue line is the unbounded protein, and all ...
0
votes
0answers
14 views

Markov model for time series, going back n periods?

I realise there are a lot of questions about markov here, but as we say in Dutch, I couldn't see the threes through the forest. I have a sequence of intervals between subsequent notes (pitches). ...
2
votes
0answers
28 views

Markov chain and process

This is not homework. Just practicing for an upcoming exam. Question is taken from a web pdf : http://www.dartmouth.edu/~chance/teaching_aids/books_articles/probability_book/Chapter11.pdf Consider ...
0
votes
1answer
23 views

How can a probability distribution P not factorize over a graph H when P satisfies the independencies implied by H

How can a probability distribution P not factorize over a graph H when P satisfies the all the global independencies implied by H? Here's an example: Let $X_1, \dots X_4$ be 4 random variables that ...
0
votes
1answer
10 views

Way to avoid predefining actions and states in reinforcement learning

Recently, I have heard of the concept of the reinforcement learning and I have got interested in it. So I decided to start a project that uses this kind of machine learning for constructing algorithms ...
2
votes
1answer
30 views

Mixture of normals, dependent on past

I have the following probability model: $(X_k|\text{PastHistory}_{k-1}, \theta_0,\theta_1,\theta_2) \sim (\pi\cdot N(\theta_1+\theta_0\cdot X_{k-1},1)+(1-\pi)\cdot N(\theta_2+\theta_0\cdot ...
0
votes
1answer
39 views

What Bayesian model should I use for posterior room identification?

I have n rooms (which can be considered as states) and a sensor on my robot which gives me a probability array of what room it is in (this array is of size n and its sum is 1). At every timestamp (the ...
0
votes
0answers
36 views

Discrete Time Markov Chain - Inventory

Let $D_n$ be the demand for an item at a store on day $n$. Suppose that $D_n$ is a sequence of independent and identically distributed (i.i.d.) random variables with probability mass function: $p_k = ...
3
votes
0answers
43 views

Continuous time Markov chain forward equation

This is a homework question and I need suggestion how to approach it. We have given the transitions $\ i\rightarrow i+1$ with rate $\lambda(i)$ where $\ i \ge 1$ $\ i\rightarrow i-1$ with rate ...
0
votes
0answers
50 views

Easy to follow tutorial on using Markov Random Fields for classifying pixels in gray-scale images

I am trying to learn how to use Markov Random Fields for classifying pixels in an image. Could someone please direct me to a simple tutorial demonstrating how this is done. The tutorial needs to ...
1
vote
1answer
108 views

Estimating probability of 2 dependent variables

I am working on a programming problem of 2 dependent variables X and Y. X is made of feature vector of size [128 x 1] and Y is the observation. My problem is that the 2 variables are dependent that ...
1
vote
0answers
29 views

is this two dimensional markov chain correct for this queueing system?

the problem that i have two single server station with no queuing space a customer goes to station 1 if it is available else it goes to station 2 if it is available or it will be lost output from ...
5
votes
1answer
119 views

Graphical models for correlation of random variables and prediction of hidden observations

I am studying about Graphical Models and I came up with a simple example but I am not sure which kind of technique (HMM, DGM, MRF) would be able to help me with that. Imagine we have three balls that ...
0
votes
0answers
26 views

Testing time homogeneous Markov chains

I am working with different transition diagrams and want to calculate the likelihood ratio statistic for testing time-homogeneous. I saw that there are already some comparable questions, but I still ...
0
votes
0answers
9 views

What to do with a markovian process with unknown path length?

I have a markovian process with an initial state distribution $p(Z_0)$; i want to predict $Z_i$ for a fixed $i$ and i observe $Z_{i+j}$. However, the catch is that $j$ is also a hidden variable with a ...
0
votes
1answer
135 views

Convergence of value iteration

Why is the termination condition of the value-iteration algorithm ( example http://aima-java.googlecode.com/svn/trunk/aima-core/src/main/java/aima/core/probability/mdp/search/ValueIteration.java ) as ...
0
votes
0answers
20 views

Markov Decision Process and its generality

My major is CS and I have a question about Markov decision process. I have been reading a book, planning with markov decision process an AI perspective. While reading it, I have a question regarding ...
0
votes
1answer
85 views

Issue in graph construction

I have a symbolic representation of time series obtained from SAX toolbox. I was wondering if it is possible to construct a graph where each node represents a unique symbol and the edges represent the ...
3
votes
1answer
53 views

In message-passing methods, what is the actual content of the messages?

In message-passing methods, factors and random variables exchange messages that typically encode marginals, but as much as I look at their formulas, I still don't understand what those messages ...
2
votes
1answer
65 views

Variational Bayes vs EP and other message-passing methods

I am trying to understand the difference between: Expectation Propagation (EP) Variational Bayes Wikipedia says: Expectation Propagation differs from other Bayesian approximation approaches ...
0
votes
1answer
61 views

Markov Switching Forecast. How can I derive this?

Consider the autoregressive model, $\left[ \begin{array}{l} y^{\ast}_t\\ x_t^{\ast} \end{array} \right] = \left[ \begin{array}{l} a_{11}\\ a_{21} \end{array} \begin{array}{l} a_{12}\\ ...
1
vote
1answer
221 views

Hidden Markov Model to predict the next state

I am learning to use HMM and I am trying to solve the following problem. There is a robot moving around the nodes in graph. The robot can move to adjacent nodes with certain probabilities. Each time ...
4
votes
1answer
59 views

If Chebyshev's upper bound gives same value as the actual probability calculation, what can we conclude?

As an example, if Chebyshev tells $P(|X-\mu|\geq k\sigma)\leq 0.25$ and the actual probability for $k=2$ is also $0.25$.
0
votes
1answer
108 views

Markov chains that do not contain all the states in the model

I am trying to understand Mixture Markov Models in order to cluster a set of sequences that do not necessarily all have the same states occurring in them. If I have several sequences that I am ...
2
votes
0answers
34 views

Using sequential observations to perform online prediction

I'm trying to perform predictions from a sequence of events. My problem is this: Data collection: Suppose you can continuously observe a person sitting in a library. You take note of every time that ...
0
votes
0answers
44 views

Performing online prediction from sequential observations

I am trying to perform some predictions from a sequence of observable events. My problem can be abstracted like this: Data collection: Suppose you can continuously observe a person sitting in a ...
2
votes
0answers
49 views

Confusion related to Gaussian Markov Random field

I was reading this paper related to Gaussian Markov Random field. I didn't get how they derived this equation from the standard multivariate gaussian distribution equation The multivariate gaussian ...
0
votes
0answers
164 views

How to estimate mean first passage times in a Markov chain?

I am working in a very large space, many billions of elements. I can calculate the transition probability between elements exactly. If the space was small, that would be enough to calculate the mean ...
5
votes
3answers
214 views

A practical example for MCMC

I was going through some lectures related to MCMC. However, I don't find a good example of how it is used. Can anyone give me a concrete example. All I can see that is they run a Markov chain and say ...
3
votes
0answers
172 views

How do sudden spikes affect hidden Markov models

I have some data that has two sudden spikes (almost like extreme masses or Dirac delta functions) within my time series data and I was wondering if that is a problem when building hidden Markov ...
1
vote
0answers
42 views

$n$-th order Markov model with $m$ conditional probabilities

Initial Problem I am working on a problem that determines the completion of projects. There are eight stages for the project to be in before completion, with the ninth stage being completion. There ...
0
votes
0answers
128 views

Estimating the number of states via Hidden Markov Models

I have time series data from an accelerometer, and was wondering if hidden markov models (which I am not too familiar with) can be used to estimate the number of states in the model? I know typically ...
3
votes
0answers
108 views

Laplace smoothing parameter choice for Markov chain transitions

Let $Y_{t}$ be the state of the process at time $t$, ${\bf P}$ be the transition matrix then: $$ {\bf P}_{ij} = P(Y_{t} = j | Y_{t-1} = i) $$ Since this is a Markov chain, this probability depends ...
1
vote
0answers
140 views

What is the relevance of bootstrapped confidence intervals on markov chain transition matrices?

Am I right in thinking that bootstrapping is just replicating your original data x amount of times? So I have a series of $n=20$ sequences of varying length containing the letters ACGTE. The ...
1
vote
0answers
66 views

Test Markov Chain properties of time series

First of all, please excuse if I don't use the proper terminology for this problem. I have a markov chain composed by two states: When in state 1 the output is drawn from an exponential ...
3
votes
1answer
83 views

Constructing a tridiagonal Markov Chain given a desired distribution

Given a desired distribution over a set of finite states, I wish to construct a tridiagonal Markov chain that would have a stationary distribution same as the desired distribution. The reason I want ...
0
votes
1answer
94 views

Significance of a 1 state Hidden Markov Model

I've been training different observation sequences to obtain different HMMs corresponding to each observed data. Something intriguing is that I get one observation sequence represented by 1 state. ...