With a discrete state-space discrete time markov chain, given a sequence of sample data $X_{1} \dots X_{n}$, I might estimate the transition probabilities $P_{ij}$ using relative frequencies. From our sampled data, if from state $i$ we transitioned to state $j$ a total of $n$ out of $m$ times then I might just estimate $\widehat{P_{ij}} = \frac{n}{m}$.

Similarly, for a continuous state-space, but discrete time markov process, what techniques exist to estimate the stochastic markov kernel if I have a sequence of real-valued sample data $X_{1} \dots X_{n}$?


1 Answer 1


Broadly speaking, time series analysis. You can get a lot of mileage without assuming anything about the conditional distribution of the Markov kernel.

For instance, if you think that the process obeys an AR(1) process (i.e. $ X_t $ is hypothesized to satisfy the difference equation $ X_t = \phi X_{t-1} + \epsilon_t $ where $ \{ \epsilon_t \}_{t=-\infty}^\infty $ is a white noise time series with variance $ \sigma^2 $) that satisfies weak stationarity, then you can estimate $ \phi $ with a variety of procedures (by solving the Yule-Walker equations, for example).


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.