Problem statement

Let $\mathbb{X} \subset \mathbb{R}^{k}$ and let $p:\mathbb{X}\times\mathbb{X}\rightarrow\mathbb{R}$ be a density kernel on $\mathbb{X}$. We assume the following model for the stochastic process \begin{equation} X_{t+1}\sim p(X_t, y)dy \quad (t\geq1), \end{equation} which defines a discrete time Markov process ${(X_t)}_{t\geq1}$ on $\mathbb{X}$. Let $X_0$ be a given initial state, and let $(y_i)_{i=1}^{T}$ be an observation, or realization, of that Markov process.

Furthermore, assume that we have observed another $n$ independent paths ${(x_{i}^{j})}_{i=1}^{T}$, where $j=1,\dots, n$, from the discrete time Markov process.

Would anyone be able to tell me what can be said about the equation \begin{equation} \frac{1}{n}\sum_{j=1}^{n}\prod_{i=1}^{T}p(x_{i-1}^{j}, y_i), \end{equation} and in particular what happens in the limit as $n\rightarrow\infty$?

This has been proposed to me as a way of estimating the likelihood, or density, of the observed path $(y_i)_{i=1}^{T}$ for the assumed model of the stochastic process, with the motivation that it propagates the uncertainties of the model, but I have been unsuccessful in understanding the equation.

I hope that this is clear enough to at least give a hint which may lead me down the proper path.

Insights (update)

I have found a paper by Stachurski et. al (2008), which give some insight for the problem.

The marginal distribution of $y$ at time $t$, $\mathbb{P}(X_t = y)$, can be written (assuming a given initial condition) as \begin{equation} \psi_{t}(y) = \int p(x,y)\psi_{t-1}(x)dx, \end{equation} and we define an estimate of this as \begin{equation} \psi_{t}^{n}(y) \equiv \frac{1}{n}\sum_{i=1}^{n}p(x_{t-1}^{i}, y), \end{equation} where ${(x_{t}^{i})}_{i=1}^{n}$ are $n$ i.i.d realizations, or observations, of the Markov process at time $t$.

By the law of large numbers we get \begin{equation} \frac{1}{n}\sum_{i=1}^{n}p(x_{t-1}^{i}, y) \rightarrow \mathbb{E}p(x_{t-1}^{i}, y) = \int p(x,y)\psi_{t-1}(x)dx = \psi_t(y), \end{equation} as $n\rightarrow\infty$ (Stachurski, 2008).

However, this only helps us get insight into estimates of the marginal probability distributions of the discrete time Markov process, and not the joint probability distributions.

We could make the assumption that ${(X_t)}_{t\geq1}$ are i.i.d (which is incorrect by the definition of the model, but lets see what happens), which would allow me to write \begin{equation} \label{eq:likelihood_1} \mathbb{P}(X_1=x_1, \dots, X_T=x_T) \approx \prod_{i=1}^{T} \psi_{i}^{n}(x_i) = \prod_{i=1}^{T} \frac{1}{n}\sum_{j=1}^{n}p(x_{t-1}^{j}, x_t), \end{equation} which looks very similar to the equation I have an interest in, the major two differences being that the product is in a different place and the way we draw ${(x_{t}^{i})}_{i=1}^{n}$.

While this helps give some insight into the problem, I am still unable to understand the equation of interest. Namely, \begin{equation} \frac{1}{n}\sum_{j=1}^{n}\prod_{i=1}^{T}p(x_{i-1}^{j}, y_i). \end{equation}


1 Answer 1


Sounds like you are you looking for the partition function for a given path and then a functional integral across those paths in the denominator to determine the probability of a given path.

  • $\begingroup$ You know have enough rep to comment anywhere. Congratulations. This is rather a comment than an answer. $\endgroup$
    – Ferdi
    Oct 6, 2017 at 8:56
  • $\begingroup$ Thank you for your answer. I am sure that you are correct, however, my foremost interest is in understanding the equation suggested. I need to understand the interpretation of it in this context, and if there is no reasonable interpretation (the equation is not meaningful) why that is. Maybe I would get further insights into the problem if you could elaborate on how to write down a partition function for the given path and a functional integral across those paths? (I do not have the proper background in physics) Keep in mind that I need to be able to make an estimate by computation. $\endgroup$
    – Kuno
    Oct 6, 2017 at 9:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.