Is any random process $Y_{t}$ such that the future is conditionally independent of the past, given the present. That is the distribution of the process only depends on where the process is, not where it has been: $$ P(Y_{t+1}=y_{t+1} |Y_t = y_{t}, Y_{t-1} = y_{t-1}, ..., Y_{1} = y_{1}) = ...
0
votes
1answer
36 views
Determine the communication classes for this Markov Chain
Say we have a Markov Chain with probability matrix
$$ P = \begin{pmatrix} 0.25 & 0.25 & 0.5 & 0 & 0 \\
0 & 0.66 & 0 & 0.33 & 0 \\
0 & 0.25 & 0.25 & 0.25 ...
2
votes
1answer
55 views
Show that the minimum time to get back to state $1$ is $(0.5)^{k-1}$
Suppose that the chain is intitially in state $1$, i.e $P(X_0 = 1) = 1$. Let $\tau$ denote the time of first returen to state $1$, i.e
$$\tau = \min\{n > 0: X_N = 1\}.$$
Show that
...
0
votes
1answer
84 views
Hidden Markov Model - Confusion
I don't know whether this is the correct forum for this but here goes:
I'm trying to implement a Hidden Markov Model to be able to predict and find the best sequence/path for a training file.
So ...
1
vote
0answers
49 views
Predicting the next value
I've recently read a blogpost where someone tries to predict the outcome of eurovision.
Quoting from the summary of the site:
Essentially, we can look at people’s voting preferences in the
...
2
votes
0answers
48 views
How does predictive model for the Eurovision Song Contest work?
I've encountered interesting prediction of Eurovision Song Contest http://mewo2.com/nerdery/2013/05/12/eurovision-2013-first-predictions/ it based on some kind of Bayesian model I assume but I don't ...
6
votes
1answer
200 views
Evaluating clusters of first-order Markov chains
I clustered my dataset of several thousand first-order Markov chains into about 10 clusters.
Is there some recommended way how I can evaluate these clusters and find out what the items in the ...
0
votes
0answers
21 views
Measuring the possibility of occurrence of tree according to a transition matrix
I have a $n\times n$ transition matrix where each value represents a node (of the tree) to node transition probability. I want to measure of possibility of occurrence of a tree according to the ...
0
votes
1answer
49 views
How to generate the most common clickstream sequence
I have logs with the following information:
date-time username view action action_data
These logs are generated from a web-application which consists of several views where the users can perform a ...
2
votes
0answers
32 views
How to compare two matrices?
I am working on Markov transition matrices. I would like to find a statistical test to compare them.
The first matrix is considered the population transition matrix and the second one is obtained by ...
0
votes
0answers
40 views
Mathematically modeling neural networks as graphical models
I am struggling to make the mathematical connection between a neural network and a graphical model.
In graphical models the idea is simple: the probability distribution factorizes according to the ...
3
votes
1answer
54 views
Intutive difference between hidden Markov models and conditional random fields
I understand that HMM are generative models, and CRF are discriminative models. I also understand how CRFs' are designed and used. What I do not understand is how they are different from HMMs'? I read ...
1
vote
0answers
30 views
Foundamental limit theorem of Markov chains with higher order chains?
I have a second order Markov chain with 4 states {A,T,C,G} (the 4 DNA nucleotides).
the transition matrix looks like this:
...
0
votes
1answer
50 views
Does a solution for the stationary distribution of a Markov chain guarantee the distribution exists?
Say we have a Markov chain with a countably infinite state space, e.g. the non-negative integers.
If we can form and solve equations for the stationary distribution {$\pi_i$}, that satisfies:
$\pi_i ...
0
votes
1answer
52 views
How to simulate a hidden Markov chain?
I want to simulate data from a 3-state hidden Markov chain with a known matrix of transition probabilities. Each state corresponds to a bivariate data with known marginals that the dependence between ...
0
votes
0answers
58 views
Is reflected random walk null or positive recurrent?
My question is whether the following Markov chain is positive or null recurrent. It's a random walk with a reflecting barrier at the origin.
Let $X_n \in \mathbb{N}$. Then the transition ...
0
votes
1answer
60 views
Markov model's first step analysis
Following is the problem:
You have five fair coins. You toss them all so that they randomly fall heads or tails. Those that fall tails in the first toss you pick up and toss. You toss again those ...
0
votes
0answers
15 views
Markov Chain on a General State Space
I am having some difficult understanding some early results of Makov Chain theory on a general state space.
We have a function (Kernel) $K:E \text{x} E \rightarrow \mathbb{R}$, and a distribution ...
1
vote
0answers
24 views
Identity in Markov Process
I want to know if my reasoning here is correct, it seems simple enough but I just want clarification (I am considering the proof that if a Markov process satisfies the detailed balance condition, then ...
0
votes
0answers
44 views
Question about infinte Markov Chains
Do 2 Markov chains $\left\{X_n\right\}^\inf_{n=0} $ and $\left\{Y_n\right\}^\inf_{n=0} $ with all of the following properties exist so that the probability for infinite n values to maintain ...
2
votes
1answer
207 views
Figuring out probabilities with Hidden Markov Models
I'm really new to statistics so sorry in advance if this question does not make sense.
Background:
I'm trying to learn about hidden Markov models and they seem interesting but I was wondering about ...
0
votes
0answers
51 views
Baum-welch algorithm: probabilities after each step
In an effort to understand machine learning, at least to some degree, I've been implementing the various algorithms to solve the three problems in a Hidden Markov Model. I've been using Rabiner's ...
1
vote
1answer
23 views
Reducible Markov Chain with a state that communicates with nothing
Lets say there is a Markov chain whose transition matrix is defined as follows
$$
P = \left( \begin{array}{cccc}
0.5 & 0.5 & 0 & 0 \\
0 & 0.25 & 0 & 0.75 \\
0 ...
0
votes
1answer
91 views
Python module to compute stationary distribution of Markov chain
Is there any module for Python that computes the stationary distribution of a Markov chain, given the generator matrix?
Thanks for any help!
10
votes
3answers
167 views
Build a path probability tree for journeys through a website
I'm currently doing analysis on a website which requires that I create a decision tree diagram showing the likely route that people take whenever they arrive on the website. I am dealing with a ...
2
votes
1answer
74 views
Is this a valid markov chain?
I have the Markov chain presented in the image. It hast 4 states S=(1,2,3,4). The transitional probabilities are all $\frac{1}{2}$ and the direction from one state to another is given by the arrows.
...
0
votes
1answer
54 views
markov chain for prediction
I am using markov chain process for prediction. I created a transition matrix using training dataset .Now I want to experiment in test dataset and want to compute the accuracy of transition matrix for ...
0
votes
1answer
39 views
The effect of number of states in markov chain process
What is the effect of increasing or decreasing the number of states in markov chain process ?
3
votes
1answer
115 views
the combination of two independent continuous time Markov chains
Assume one variable $x$ has two states 0 and 1, $x$ changes between 0 and 1 following a continuous time Markov chain. The transition probability is represented as matrix $P$ and the time sojourning on ...
1
vote
0answers
11 views
Models for learning acquistion process
Imagine such a memory test for a mice. The mice performs an experiment $E$ with two possible issues $0$ (failure) and $1$ (success). If the mice gets $1$ it is "rewarded", if it gets $0$ it is ...
9
votes
2answers
198 views
Numeric solvers for stochastic differential equations in R: are they any?
I'm looking for a general, clean and fast (i.e. using C++ routines) R package for simulating paths from a non-homogeneous nonlinear diffusion like (1) using the Euler-Maruyama scheme, the Milstein ...
1
vote
1answer
108 views
Using a Markov Chain to find the limiting probability?
Let's say a website makes available only one of three online quizzes A, B and C, daily.
If the majority of visitors pass the quiz then the next day the website will randomly publish either quiz A, B, ...
0
votes
1answer
123 views
How to define initial probabilities for HMM?
HI This is first time I was reading about HMM, however I have read so many articles on web, but two things where I am confused are:
How to decide number of Hidden States (although HMM says we don't ...
1
vote
0answers
40 views
Hidden Markov model alternative that allows dependence on previous n states
I am working on a prototype framework.
Basically I need to generate a model or profile for each individual's lifestyle based on some sensor data about him/her, such as GPS, motions, heart rate, ...
1
vote
0answers
26 views
Markov chain from Poisson: null or positive recurrent?
Let $K_t$ be a Poisson process with rate $1$ and $X_n=K_n-n$ $, \ \ \ n\in \mathbb{N}$
Now I want to determine whether it is null or positive recurrent, we already know it is recurrent.
I ...
0
votes
0answers
56 views
0
votes
0answers
37 views
Form field identification
I am working on a problem in which given a form (mostly scanned) one needs to automatically detect all the fields and their corresponding values from it. The information that I have is location of ...
2
votes
2answers
70 views
Calculating transition probabilities for Ehrenfest diffusion
Two containers $A$ and $B$ are placed adjacently to each other and gas passes through a small aperture joining them.
A total of $N(>1)$ molecules is distributed among the containers.
...
0
votes
0answers
51 views
how to calculate future state probability if present probability is given by Markov chain
There is 50 elements transmitted from transmitter to receiver in 1st time where the probability of each element is not received by receiver is 0.1 and it will be received by receiver with probability ...
1
vote
0answers
58 views
Can I adapt a MCMC proposal using a parallel chain?
I am running two MCMC chains (say chain A and chain B) in parallel, using the Metropolis-Hastings algorithm with acceptance probability:
$P(accept\ x_t) = \min\{1, f(x_t)/f(x_{t-1})\}$.
I would like ...
1
vote
0answers
42 views
Method of Additional Events for the two-state Markov Process
I'm trying to understand a research paper but am facing a few difficulties, any help would be really appreciated.
A three state markov process with transition probabilities for states being:
0 --> ...
2
votes
1answer
93 views
Recurrence definition for a Markov chain
We define a state i to be recurrent if $\sum\limits_{n=0}^\infty P(X_n=i,X_k \neq i$ for $1\leq k < n | X_0=i)$=1.
Why do we take infinite series over the probability? Why don't we define ...
0
votes
1answer
31 views
What values can the process $X_n$ take when $P(Y = k) = a_k = 0$ for $k \geq 2$
A newspaper uses one ton of newsprint every day. It buys its newsprint from a local distributor. This ditributor supplies the newsprint in one-ton rolls at the cheapest price, but unfortunatley its ...
5
votes
1answer
136 views
Is there a standard name for probabilistic graphical models like this?
I have some data I need to analyze, and some prior knowledge I'd like to apply. I have discrete time points, and noisy, continuous outputs $x(t)$ that I observe at those time points, and would like ...
0
votes
0answers
69 views
What is forward and backward probability? [closed]
I am trying to understand Forward-Backward probability.
I understood Transition Probability, Emission Probability, Viterbi. But getting confused with Forward-Backward. I was trying to work out the ...
13
votes
6answers
496 views
Check memoryless property of a Markov chain
I suspect that a series of observed sequences are a Markov chain...
$$X=\left(\begin{array}{c c c c c c c}
A& C& D&D & B & A &C\\
B& A& A&C & A&D &A\\
...
2
votes
1answer
84 views
Markov chain model likelihood ratio test
Suppose I am using two Markov Chain Models, one with order $k=1$ and a second one with order $k=2$. I am "reducing" the higher order model to a $k=1$ model in order to have easier calculation ...
2
votes
0answers
42 views
Markov random field and iterated condition mode
I have spent a lot of time studying MRF (applied to images) but still can't grasp the idea. Could you please clarify these ideas:
What is the clique potential? What is a clique in image, and do they ...
3
votes
1answer
97 views
Stationary matrix given a transition matrix
I am given the following transition matrix
$$P= \pmatrix{ 1-\alpha & \alpha \\ \beta & 1-\beta}, \ \alpha,\beta \in (0,1)$$
with the states $S=\{1,2\}$.
I want to determine the stationary ...
1
vote
1answer
157 views
Intuitive explanation for periodicity in Markov chains
Can someone explain me in a intuitive way what the periodicity of a Markov chain is?
It is defined as follows:
For all states $i$ in $S$
$d_i$=gcd$\{n \in \mathbb{N} | p_{ii}^{(n)} > 0\} =1$
...
3
votes
1answer
101 views
Proof of theorem on recurrent states and its equivalence class
A theorem states the following:
Theorem
if $i \in S$ is a state which is recurrent, then every state in the equivalence class of $i$ $(\ K(i) \ )$ is recurrent.
Additional information on the ...
