# Questions tagged [markov-process]

A stochastic process with the property that the future is conditionally independent of the past, given the present.

1,189 questions
Filter by
Sorted by
Tagged with
53 views

### How could I estimate a transition probability matrix that varies over time?

I have multiple Markov chains with twelve states. I want to estimate a transition probability matrix for each time point (except for the last time point) that can vary over time using all Markov ...
8 views

### Attribution modelling using First and Higher-Order Markov Chains

The crux of my question is as follows: Would a higher-order Markov model produce a different result than a first-order Markov model when used for Channel Attribution modelling? Once the transition ...
5 views

### Main event time prediction based on different sub events

As the title says, I want to predict the time (with a wide error range) of a main event’s first occurrence based on previous sub events that are vary in importance. These previous ‘predictor’ events ...
25 views

### stationary distribution of a continuous time markov chain

With a certain rate $R$ balls fall into a box. There is no limit to the number of balls the box can hold, but each ball has a rate $\gamma$ to leave the box and when two balls hit each other they ...
23 views

### Numerically stable computation of the variance [duplicate]

Suppose I've sampled $x_0,\ldots,x_{n-1}$ and want to calculate the variance of these samples. What is a good (numerically stable) algorithm for this? And does the answer change, if we impose ...
• 419
45 views

### What kind of transition probability matrix indicates dependence/independence?

Suppose we have two discrete random variables $X$ and $Y$, both of which take values from $\{1,2,...,k\}$. $Y$ is generated from $X$ via a transition probability matrix (also known as the stochastic ...
• 189
25 views

### How are differential equations related to Markov processes?

How can I classify the two terms differential equation and Markov process? Here are a few questions that I ask myself: Is a Markov process a superset of differential equation or vice versa? Do ...
• 157
10 views

### invariant distribution in discrete time markov chain and continuous time markov chain

The jump chain of a continuous time markov chain is a discrete time markov chain. I know that the existence of an invariant distribution for one chain does not imply the existence of an invariant ...
29 views

### Determine recurrent states via First Return Probability

I have the following question. It's about transient and recurrent states in Markov Chains. I know when a state is one or the other, but there is one thing I can't figure out or understand. We have ...
• 11
1 vote
25 views

• 121
108 views

### How To Estimate Markov Chain Probabilities in Real Life? [duplicate]

A question that I have always wondered about is that how are the Transition Probabilities within a Markov Chain estimated in real-world applications? I tried to learn more about this online and found ...
• 7,028
1 vote
40 views

### Does large mutual information (between observations and parameter) imply the existence of a good estimator?

This question concerns the standard setting for applying Fano's inequality to derive minimax bounds for a parameter estimation problem. The goal is to estimate a parameter described by a random ...
• 163
62 views

### Training an autoregressive model using multiple independent sequences

I have a number of independent observed sequences that I believe are generated by same underlying process. Is it possible to extend linear autoregression and learn the optimal transition matrix for ...
• 81
22 views

### Bayes update rule studied as an operator

The bayes rule can be understood as a nonlinear map from the space of probability measures to itself. Are there any reference/books which study it from this perspective? For eg. can we say something ...
1 vote
47 views

• 21
1 vote
52 views

### How to use Stirling's formula of n! in this probability computations in random walk?

I want to compute $\binom{2n}{n} p^n (1-p)^n = \frac{(2n)!}{n!n!}(p(1-p))^n, n=1,2,3...$ By using an approximation, due to Stirling, which asserts $n! \sim n^{(n +\frac12)}e^{-n}\sqrt{2\pi}$ Where ...
• 1,140
For a Markov chain $\{X_n, n \geqslant 0 \}$ with transition probabilities $P_{i,j},$ consider the conditional probability that $X_n =m$ given that the chain started at time 0 in state i and has not ...