A stochastic process with the property that the future is conditionally independent of the past, given the present.

learn more… | top users | synonyms (1)

0
votes
0answers
18 views

Markov chain approximation of continous distribution

I'm trying to build a model to describe the evolution of a zero-inflated continuous variable. Having no clear idea of what model to use, i though about using a Markov chain. However it vould require ...
1
vote
0answers
16 views

How do I check or validate the RBM (Restricted Boltzmann Machine) Model?

I'm trying to implement RBM, then i used play tennis case to test the rbm. I've tried autoencoder before, and the result was good. Actually, I confuse with the function of RBM it self, i think it ...
0
votes
0answers
13 views

Joint Markov Chain (Two Correlated Markov Processes)

I have two Markov Chains, and they exhibit some correlation between them. For instance, when Chain A moves to state i, there is a high likelihood that Chain B moves to state j. How would I go about ...
0
votes
0answers
15 views

is there any R package for calculating transition probabilities for multi order markov chain?

I need to estimate the transition probabilities of a user whose k recent selections were x1, . . . ,xk and will select the item x' next. I know to calculate first order transition probabilities but ...
3
votes
1answer
20 views

Markov chain with conditions

I have a data that is being modelled through the continuous time Markov chain with discrete state space. The model is simple: only 4 states. However, I have an additional condition imposed on the ...
0
votes
0answers
23 views

Real Data Sets Examples

I have a set of observations $\mathcal{Y} = {Y_1, \cdots, Y_T}$. I am running EM algorithm to fit the observations to the following Hidden Markov Model $$A = [a_{ij}]_{N \times N}, a_{ij} = P(X_{k+1} ...
0
votes
0answers
4 views

Find the expected frequency of some state in a state sequence of length N given a transition matrix M

I can represent stochastically-articulated sequences of states using a transition matrix M where a given entry in cell (i,j) corresponds to the probability of state j given that the current (or, most ...
1
vote
0answers
10 views

Given an transition matrix what is the likelihood an observed markov chain was derived from this matrix

To give a bit of background, I'm creating a MLE of a transition matrix from a set of empirical data. I'm then creating a simulation of the system that also produces a markov chain. I am looking for a ...
0
votes
0answers
26 views

How do I use Hidden Markov Model Viterbi algorithm for sequence labeling?

with my current small experience of HMM. Given that i have some patterns (sequence of interest for example gestures or words in spoken language) if i need to use HMM for sequence classification ...
0
votes
0answers
11 views

multiple, related, time-series applied to Statistical Process Control

I tried looking online (google), searching in stack-overflow and cross-validated, and just looking through "R" documentation for the answer, but either I am not seeing it, or I don't know how to tell ...
1
vote
0answers
20 views

MLE of CTMC parameters

Let the data set be $$D = \{(s_0, t_0), (s_1, t_1), ..., (s_{N-1}, t_{N-1})\}$$ where $N=|D|$. Each $s_i$ is a state from the state space $S$ and during the time $[t_i,t_{i+1}]$ the chain is in state ...
1
vote
0answers
27 views

How I can use markovchain package when I have many sequences? [closed]

I use R program and markovchain-package. I have file where I row I one person. In row is a list, where countries that person has vited and what order for example Person one: USA, Canada, Germany ...
0
votes
0answers
30 views

Markov Modeling, My repair is constant and not memoryless

I want to calculate a markov model, but there is a problem; my repairing transition is not memoryless and it's constant for every time that it will happen.(Markov model consider all of transitions are ...
6
votes
1answer
113 views

Expected number of times you spent in a state of an absorbing markov chain, given the eventual absorbing state

It's well known that, if $Q$ is the matrix of transient state transition probabilities, and $$ N = \sum_{n=0}^{\infty} Q^n = (I - Q)^{-1}$$ then $N_{ij}$ describes the expected number of times the ...
2
votes
1answer
54 views

Calculating probability for a continuous time markov chain

I can do question 1-3 no problem however I cant follow the steps to get the answer to part 4. Can anyone here explain the steps I need to go through in more detail?
0
votes
0answers
5 views

Measure of incoming communication

Given a standard Markov Chain on discrete time and finite statespace, represented by a matrix $M$, with $\sum_{j=1}^n m_{ij}=1$ I have a certain absorbing state k, where the incoming communication is ...
0
votes
1answer
31 views

how to calculate the following conditional probability

There are two events involved, say event A and event B. I want to know the probability of event B conditioned on the event A. The relation between the two events are as follows. We can not talk about ...
0
votes
0answers
10 views

Texture Synthesis

I'm looking to synthesise some textures that look like natural ceramic tiles in Matlab. This isn't something I've tried before so I was wondering what the best techniques to use would be or if there's ...
3
votes
0answers
205 views

How to understand Gibbs distribution

I have a graph model such as Following the Hammersley–Clifford theorem describes that Markov random fields exhibit a Gibbs distribution with an energy function as follows: $$P(x)=\frac ...
0
votes
0answers
25 views

Calculating joint density function of Brownian motion

I read in my book today regarding the calculation of the joint density function of a brownian motion process and it went as follows: If we define $X(t)$ as a Brownian motion process with mean $0$ and ...
0
votes
1answer
32 views

Transition Probability Matrix with zeros

A markov chain model TPM for daily rainfall is given by: [0.4 0.6 0.0; 0.2 0.6 0.2; 0.0 0.7 0.3]; Thus, it is not possible to pass directly from state 1 to ...
0
votes
0answers
55 views

How can I infer the value of multiple dependent continuous random variables in conjunction with discriminative learners?

I have 2 continuous random variables V1, V2 which are dependent. I want to infer each of their values based on: The value of ...
0
votes
0answers
32 views

Binning by standard deviations

Quick question: I came across a fairly respected source on running Markov Chain Monte Carlo for bayesian statistics in ...
0
votes
0answers
18 views

Estimating transition probabilities from the sequences without repeated states

Is it possible to estimate transition probabilities of the continuous process if we observe only the first occurence of the state? For example, if the real transition sequence was 1 - 5 seconds; 2 - ...
2
votes
0answers
34 views

Markov models that with several active states

Are there any Markov-like models that can have several active states? So say if trying to determine (the chance) when the person will wake up based on two variables (weather and the time the person ...
0
votes
1answer
25 views

Can reinforcement learning be used as path guidance?

I have an online path guidance system that learns from a set of past experiences(trajectories) to provide a guidance for the user on how to cover the given space in the best way, adopting to whatever ...
0
votes
0answers
34 views

anomaly detection with Markov chain

The paper uses a simple technique to detect intrusions in computer systems. I will briefly explain it and ask a question: The paper proposes a simple 1-order Markov chain modelling approach to detect ...
-1
votes
1answer
42 views

markov chain - probability question

Transition matrix has been written like that; $$\mathcal P = \begin{bmatrix} 1/3 & 0 & 2/3 \\ 1/3 & 1/3 & 1/3 \\ 0 & 0 & 1 \end{bmatrix}$$ the initial vector is that ...
1
vote
0answers
30 views

How to estimate Markov chain transition probabilities with partially observed data?

Suppose that we have a time-homogeneous discrete-time Markov chain $(X_n)$. We want to estimate the transition probabilities $p_{ij} = \mathbb{P}[X_{n+1} = j \mid X_n = i]$. In the case when we have ...
1
vote
0answers
51 views

Measuring effectiveness of marketing through attribution analysis [closed]

My data(dataframe in R) looks like this:The data is ordered by CustomerName and then TimeofEvent. ...
3
votes
0answers
45 views

Markov decision process in R for a song suggestion software?

We have a music player that has different playlists and automatically suggests songs from the current playlist I'm in. What I want the program to learn is, that if I skip the song, it should decrease ...
0
votes
0answers
5 views

Identifying sequences of behavioral interactions between multiple individuals

I'm wondering if anyone might have some novel insights as to the best way to analyze the following data. It's a problem I've been thinking about in the back of my mind for a while, so I thought that ...
2
votes
1answer
67 views

Markov chains with a stationary distribution but no limiting distribution

I am trying to intuitively reconcile the following statement, read from "Probability, Markov Chains, and Queues": ...
2
votes
1answer
40 views

First-order Discrete Markov Chain with time lag

I want to estimate the first-order transition matrix of a sequence in discrete time, e.g. $$ s = 1,0,1,0,1,1,0,1,0,0, \dots$$ but states are not evenly spaced in time. So that even if $s_{t=1} = 1$ ...
0
votes
0answers
8 views

Estimate conditional probability of connected variables

I want to model the following probability function $p(x_i|\mathcal{N}_{x_i})$, where $\mathcal{N}_{x_i}$ is the set of the variables $x_j$ conneced to $x_i$ given a specified undirected graph ...
0
votes
0answers
17 views

Degrees of Freedom for Inhomogeneous Markov Chains

Suppose I am given a time-inhomogeneous Markov chain. For simplicity, let's assume we have 5 (time) positions, and 4 states (A,B,C,D). How can I compute the degrees of freedom for a first order model ...
0
votes
1answer
44 views

What is the expected amount of time it takes to first observe a triple for multiple dice rolls? [duplicate]

Assuming we have a fair die, we toss a die multiple times. Also assuming that a triple is defined as when we have three rolls in a roll that result in the same number, and that the rolls are ...
4
votes
1answer
52 views

Analyzing output in MCMC

I am using emcee to do inference on some data. I am trying to fit my data to a line of equation $ y = mx + b $. ...
4
votes
1answer
47 views

Given two absorbing Markov chains, what is the probability that one will terminate before the other?

I have two different Markov chains, each with one absorbing state and a known starting position. I want to determine the probability that chain 1 will reach an absorbing state in fewer steps than ...
0
votes
0answers
25 views

Limiting distribution of a Markov chain?

I have the problem below. There are n identical machines. They are all operational at time 0. The lifetime of each one is an exponential random variable with rate L. There are r repairmen (1 ≤ r ≤ ...
0
votes
0answers
11 views

What are good tutorial on Weighted Finite Automata?

I would especially appreciate papers, books or tutorials with source code already available. Currently I'm reading "Spectral Learning Techniques for Weighted Automata, Transducers, and Grammars" by ...
1
vote
1answer
32 views

Estimating Markov Switching Probit

I attempt to fit the following probit model to a time series where we observe the binary variable $R_{t}$ and another variable $X_{t}$, a latent unobserved variable $y^{*}_{t}$ and a state variable ...
-1
votes
1answer
23 views

Proof of Markov Chain property

Suppose that $X_n$ is a Markov Chain.Then for $m,n \in N$ such that $m<n$ $Pr[X_n=j_n|X_m=j_m,X_{m-1}=j_{m-1},...=X_0=j_0]=Pr[X_n=j_n|X_m=j_m]$ When proving for n=3,m=1 case we have to show ...
1
vote
1answer
25 views

Conditional transition matrix (consistent estimates)

Is there a model/technique that is able to estimate transition matrix (which would be consistent, i.e. sums of their rows would be always 1) conditional on some continuous variable X? Let's say I ...
0
votes
0answers
18 views

Distribution of the duration of a markov-process in a specified state during a specified time

I have a continuous time markov chain with two states $A$ and $B$. The transition rate $A\rightarrow B$ is $\lambda$ and $B\rightarrow A$ is $\mu$. Imagine that $P(X{t_0}=A)=1$ (the process starts in ...
0
votes
0answers
19 views

disease progression R markov chains

hello i have a dataframe, some of the columns include: remission,height,weight,time from diagnosis, age ethnicity, age, patient id remission is 1 or 0 just to be clear i want to fit an appropriate ...
0
votes
0answers
42 views

Subsetting dataframe conditions on factor (binary) column (vector in R)

I have a sequence of 1/0's indicating if patient is in remission or not. Assume the records of remission or not were taken at discrete times. How can I check the Markov property for each patient, ...
1
vote
1answer
34 views

r markov chain markov property on binary variable, discrete time

i have a sequence of 1/0's indicating if patient is in remission or not, assume the records of remission or not were taken at discrete times, how can i check the markov property for each patient, ...
1
vote
1answer
135 views

Fit and evaluate a second order transition matrix (Markov Process) in R?

I already built 1 first order discrete state Markov Chain model. It was built with R using the function 'markovchainFit()' in ...
0
votes
0answers
21 views

N-gram learning vs stochastic learning

I'm interested in comparing the differences in learning in n-grams and gradient-based learning (in my case with neural networks), particularly in the context of language modelling with the two classes ...