How to estimate the kernel of a Markov process with continuous state-space, from a finite sample?

Question

With a discrete state-space discrete time markov chain, given a sequence of sample data $X_{1} \dots X_{n}$, I might estimate the transition probabilities $P_{ij}$ using relative frequencies. From our sampled data, if from state $i$ we transitioned to state $j$ a total of $n$ out of $m$ times then I might just estimate $\widehat{P_{ij}} = \frac{n}{m}$.

Similarly, for a continuous state-space, but discrete time markov process, what techniques exist to estimate the stochastic markov kernel if I have a sequence of real-valued sample data $X_{1} \dots X_{n}$?

Answer 1

Broadly speaking, time series analysis. You can get a lot of mileage without assuming anything about the conditional distribution of the Markov kernel.

For instance, if you think that the process obeys an AR(1) process (i.e. $ X_t $ is hypothesized to satisfy the difference equation $ X_t = \phi X_{t-1} + \epsilon_t $ where $ \{ \epsilon_t \}_{t=-\infty}^\infty $ is a white noise time series with variance $ \sigma^2 $) that satisfies weak stationarity, then you can estimate $ \phi $ with a variety of procedures (by solving the Yule-Walker equations, for example).