# Questions tagged [markov-process]

A stochastic process with the property that the future is conditionally independent of the past, given the present.

1,236 questions
Filter by
Sorted by
Tagged with
32 views

### A recurrent Markov Chain implies its k-step version is also recurrent?

I am curious about whether a Markov Chain $X_n$ is recurrent implies that for any $k > 0$, $X_{kn}$ is also recurrent. Here are my observations. If $X_n$ is transient, $X_{kn}$ must be transient by ...
2k views

### Infinite dice roll probability

The following is an interview question: Two players A and B play a game rolling a fair die. If A rolls a 1, they immediately reroll, and if the reroll is less than 4 then A wins. Otherwise, B rolls. ...
• 501
307 views

### Deriving the Distribution of Markov Chain Times

I am interested in learning how to derive the probability distributions for the Time to Absorption in Markov Chains (Discrete and Continuous). In the past, I have usually done one of the following: ...
1 vote
82 views

### About paper "Deep Unsupervised Learning using Nonequilibrium Thermodynamics"

Here is the paper link related to the question from my title. In Appendix B, it computes the entropy of $p(X^T)$ and says "By design, the cross entropy to $\pi(x^t)$ is constant under our ...
1 vote
58 views

• 111
11 views

99 views

### Number of configurations in which you can roll 2 6's in a row

A colleague at work gave me a problem to think about. If you throw $n$ dice, how many configurations are there where you get at least 2 6's in a row? I worked a lot with Pascal's triangle and in the ...
30 views

### Combination of a discrete and a continuous Markov Chain in a MCMC

Recently, I've been questioning myself on the possibility of combining a discrete update and a continuous update on a single MCMC. Stephens (2000) in Algorithm 3.2 runs the process for a fixed amount ...
• 1
Given that the current sample location is $x\in[0,1)^d$, I would like to take the next sample $y$ as $$y=x+b(x)\Delta t+\sigma(x)\sqrt{\Delta t}\xi\;;\;\;\;\xi\sim\mathcal N_{0,\:I_d}\tag1$$ for some ...