Is any random process $Y_{t}$ such that the future is conditionally independent of the past, given the present. That is the distribution of the process only depends on where the process is, not where it has been: $$ P(Y_{t+1}=y_{t+1} |Y_t = y_{t}, Y_{t-1} = y_{t-1}, ..., Y_{1} = y_{1}) = ...
0
votes
0answers
23 views
Significance of a 1 state Hidden Markov Model
I've been training different observation sequences to obtain different HMMs corresponding to each observed data. Something intriguing is that I get one observation sequence represented by 1 state. ...
1
vote
1answer
41 views
Energy function of RBM
The Hammersley-Clifford Theorem tells us that the distribution of a RBM must be Gibbs since it is Markov Random Field, but how to prove that its energy function must be of the form:
$$E = ...
3
votes
1answer
34 views
Why is a 'Markov Random Field' a field?
In particular, I'm not seeing how a commutative group is defined on a graph, nor addition and multiplication.
So why is it a field?
1
vote
1answer
61 views
Confusion about hidden Markov model
I've gone through Hidden Markov models (HMM) for the past few months. However there are a few things that are confusing.
The set up is simple: I have to model some human gestures such as walking, ...
2
votes
1answer
73 views
Predicting the next value
I've recently read a blogpost where someone tries to predict the outcome of eurovision.
Quoting from the summary of the site:
Essentially, we can look at people’s voting preferences in the
...
2
votes
0answers
57 views
How does predictive model for the Eurovision Song Contest work?
I've encountered interesting prediction of Eurovision Song Contest http://mewo2.com/nerdery/2013/05/12/eurovision-2013-first-predictions/ it based on some kind of Bayesian model I assume but I don't ...
0
votes
0answers
23 views
Measuring the possibility of occurrence of tree according to a transition matrix
I have a $n\times n$ transition matrix where each value represents a node (of the tree) to node transition probability. I want to measure of possibility of occurrence of a tree according to the ...
2
votes
0answers
39 views
How to compare two matrices?
I am working on Markov transition matrices. I would like to find a statistical test to compare them.
The first matrix is considered the population transition matrix and the second one is obtained by ...
0
votes
0answers
46 views
Mathematically modeling neural networks as graphical models
I am struggling to make the mathematical connection between a neural network and a graphical model.
In graphical models the idea is simple: the probability distribution factorizes according to the ...
1
vote
0answers
34 views
Foundamental limit theorem of Markov chains with higher order chains?
I have a second order Markov chain with 4 states {A,T,C,G} (the 4 DNA nucleotides).
the transition matrix looks like this:
...
3
votes
1answer
79 views
Intutive difference between hidden Markov models and conditional random fields
I understand that HMM are generative models, and CRF are discriminative models. I also understand how CRFs' are designed and used. What I do not understand is how they are different from HMMs'? I read ...
0
votes
1answer
40 views
Determine the communication classes for this Markov Chain
Say we have a Markov Chain with probability matrix
$$ P = \begin{pmatrix} 0.25 & 0.25 & 0.5 & 0 & 0 \\
0 & 0.66 & 0 & 0.33 & 0 \\
0 & 0.25 & 0.25 & 0.25 ...
0
votes
1answer
61 views
How to simulate a hidden Markov chain?
I want to simulate data from a 3-state hidden Markov chain with a known matrix of transition probabilities. Each state corresponds to a bivariate data with known marginals that the dependence between ...
0
votes
1answer
57 views
Does a solution for the stationary distribution of a Markov chain guarantee the distribution exists?
Say we have a Markov chain with a countably infinite state space, e.g. the non-negative integers.
If we can form and solve equations for the stationary distribution {$\pi_i$}, that satisfies:
$\pi_i ...
0
votes
1answer
67 views
Markov model's first step analysis
Following is the problem:
You have five fair coins. You toss them all so that they randomly fall heads or tails. Those that fall tails in the first toss you pick up and toss. You toss again those ...
0
votes
0answers
62 views
Is reflected random walk null or positive recurrent?
My question is whether the following Markov chain is positive or null recurrent. It's a random walk with a reflecting barrier at the origin.
Let $X_n \in \mathbb{N}$. Then the transition ...
0
votes
0answers
16 views
Markov Chain on a General State Space
I am having some difficult understanding some early results of Makov Chain theory on a general state space.
We have a function (Kernel) $K:E \text{x} E \rightarrow \mathbb{R}$, and a distribution ...
1
vote
0answers
27 views
Identity in Markov Process
I want to know if my reasoning here is correct, it seems simple enough but I just want clarification (I am considering the proof that if a Markov process satisfies the detailed balance condition, then ...
0
votes
0answers
44 views
Question about infinte Markov Chains
Do 2 Markov chains $\left\{X_n\right\}^\inf_{n=0} $ and $\left\{Y_n\right\}^\inf_{n=0} $ with all of the following properties exist so that the probability for infinite n values to maintain ...
0
votes
0answers
111 views
Baum-welch algorithm: probabilities after each step
In an effort to understand machine learning, at least to some degree, I've been implementing the various algorithms to solve the three problems in a Hidden Markov Model. I've been using Rabiner's ...
1
vote
1answer
23 views
Reducible Markov Chain with a state that communicates with nothing
Lets say there is a Markov chain whose transition matrix is defined as follows
$$
P = \left( \begin{array}{cccc}
0.5 & 0.5 & 0 & 0 \\
0 & 0.25 & 0 & 0.75 \\
0 ...
0
votes
1answer
61 views
markov chain for prediction
I am using markov chain process for prediction. I created a transition matrix using training dataset .Now I want to experiment in test dataset and want to compute the accuracy of transition matrix for ...
0
votes
1answer
41 views
The effect of number of states in markov chain process
What is the effect of increasing or decreasing the number of states in markov chain process ?
3
votes
1answer
126 views
the combination of two independent continuous time Markov chains
Assume one variable $x$ has two states 0 and 1, $x$ changes between 0 and 1 following a continuous time Markov chain. The transition probability is represented as matrix $P$ and the time sojourning on ...
1
vote
0answers
11 views
Models for learning acquistion process
Imagine such a memory test for a mice. The mice performs an experiment $E$ with two possible issues $0$ (failure) and $1$ (success). If the mice gets $1$ it is "rewarded", if it gets $0$ it is ...
1
vote
1answer
117 views
Using a Markov Chain to find the limiting probability?
Let's say a website makes available only one of three online quizzes A, B and C, daily.
If the majority of visitors pass the quiz then the next day the website will randomly publish either quiz A, B, ...
1
vote
0answers
54 views
Hidden Markov model alternative that allows dependence on previous n states
I am working on a prototype framework.
Basically I need to generate a model or profile for each individual's lifestyle based on some sensor data about him/her, such as GPS, motions, heart rate, ...
1
vote
0answers
27 views
Markov chain from Poisson: null or positive recurrent?
Let $K_t$ be a Poisson process with rate $1$ and $X_n=K_n-n$ $, \ \ \ n\in \mathbb{N}$
Now I want to determine whether it is null or positive recurrent, we already know it is recurrent.
I ...
0
votes
0answers
57 views
0
votes
1answer
63 views
How to generate the most common clickstream sequence
I have logs with the following information:
date-time username view action action_data
These logs are generated from a web-application which consists of several views where the users can perform a ...
0
votes
1answer
101 views
Python module to compute stationary distribution of Markov chain
Is there any module for Python that computes the stationary distribution of a Markov chain, given the generator matrix?
Thanks for any help!
0
votes
0answers
37 views
Form field identification
I am working on a problem in which given a form (mostly scanned) one needs to automatically detect all the fields and their corresponding values from it. The information that I have is location of ...
0
votes
0answers
55 views
how to calculate future state probability if present probability is given by Markov chain
There is 50 elements transmitted from transmitter to receiver in 1st time where the probability of each element is not received by receiver is 0.1 and it will be received by receiver with probability ...
1
vote
0answers
63 views
Can I adapt a MCMC proposal using a parallel chain?
I am running two MCMC chains (say chain A and chain B) in parallel, using the Metropolis-Hastings algorithm with acceptance probability:
$P(accept\ x_t) = \min\{1, f(x_t)/f(x_{t-1})\}$.
I would like ...
1
vote
0answers
43 views
Method of Additional Events for the two-state Markov Process
I'm trying to understand a research paper but am facing a few difficulties, any help would be really appreciated.
A three state markov process with transition probabilities for states being:
0 --> ...
0
votes
1answer
99 views
Hidden Markov Model with MFCC coefficients
I don't know whether this is the correct forum for this but here goes:
I'm trying to implement a Hidden Markov Model to be able to predict and find the best sequence/path for a training file.
So ...
0
votes
0answers
78 views
What is forward and backward probability? [closed]
I am trying to understand Forward-Backward probability.
I understood Transition Probability, Emission Probability, Viterbi. But getting confused with Forward-Backward. I was trying to work out the ...
5
votes
1answer
142 views
Is there a standard name for probabilistic graphical models like this?
I have some data I need to analyze, and some prior knowledge I'd like to apply. I have discrete time points, and noisy, continuous outputs $x(t)$ that I observe at those time points, and would like ...
2
votes
0answers
45 views
Markov random field and iterated condition mode
I have spent a lot of time studying MRF (applied to images) but still can't grasp the idea. Could you please clarify these ideas:
What is the clique potential? What is a clique in image, and do they ...
2
votes
1answer
87 views
Markov chain model likelihood ratio test
Suppose I am using two Markov Chain Models, one with order $k=1$ and a second one with order $k=2$. I am "reducing" the higher order model to a $k=1$ model in order to have easier calculation ...
1
vote
1answer
182 views
Intuitive explanation for periodicity in Markov chains
Can someone explain me in a intuitive way what the periodicity of a Markov chain is?
It is defined as follows:
For all states $i$ in $S$
$d_i$=gcd$\{n \in \mathbb{N} | p_{ii}^{(n)} > 0\} =1$
...
3
votes
1answer
106 views
Stationary matrix given a transition matrix
I am given the following transition matrix
$$P= \pmatrix{ 1-\alpha & \alpha \\ \beta & 1-\beta}, \ \alpha,\beta \in (0,1)$$
with the states $S=\{1,2\}$.
I want to determine the stationary ...
2
votes
1answer
75 views
Is this a valid markov chain?
I have the Markov chain presented in the image. It hast 4 states S=(1,2,3,4). The transitional probabilities are all $\frac{1}{2}$ and the direction from one state to another is given by the arrows.
...
0
votes
0answers
103 views
Transform higher order Markov Chain to first order
I want to reduce a higher $kth$ order Markov Chain to a first order Markov Chain. As proposed in literature, this is possible to have easier possibilities to work with the first order Markov Model ...
3
votes
1answer
102 views
Proof of theorem on recurrent states and its equivalence class
A theorem states the following:
Theorem
if $i \in S$ is a state which is recurrent, then every state in the equivalence class of $i$ $(\ K(i) \ )$ is recurrent.
Additional information on the ...
2
votes
1answer
71 views
Random walk on $\mathbb{Z}$
Let $X_n=\sum\limits_{k=1}^n Y_k = X_{n-1} + Y_n$ and $X_0=0$, $(Y_n)$ i.i.d with
$P(Y_n=1)=p=1-P(Y_n=-1)=1-q, p \in (0,1)$, $(X_n)$ is a random walk on $\mathbb{Z}$.
Why is this Markov chain ...
10
votes
3answers
172 views
Build a path probability tree for journeys through a website
I'm currently doing analysis on a website which requires that I create a decision tree diagram showing the likely route that people take whenever they arrive on the website. I am dealing with a ...
1
vote
0answers
62 views
Nelson-Aalen estimator in discrete time
I was wondering whether the Nelson-Aalen estimator can be applied to discrete-time multi-state data (in months). I ask this because it seems that literature tends to present it only in the ...
2
votes
1answer
98 views
Recurrence definition for a Markov chain
We define a state i to be recurrent if $\sum\limits_{n=0}^\infty P(X_n=i,X_k \neq i$ for $1\leq k < n | X_0=i)$=1.
Why do we take infinite series over the probability? Why don't we define ...
3
votes
1answer
94 views
What is “symmetric property” for stationary distribution
I have the one step transition matrix
$$\pmatrix{0 & \alpha & 0 & \beta \\ \alpha & 0 & \beta & 0 \\ 0 & \beta & 0 & \alpha \\ \beta & 0 & \alpha & 0 ...
