# Differential Equations as Generative Models

I wonder, if we can say stochastic differential equations are generative models. I usually think about the Kalman filtering for example, we fix a discrete-time evolution equation of a certain object, which is a differential equation in continuous domain, and then use this model as a generative model to derive inference algorithms in a Bayesian setting.

Then, can we say that all differential equations involving noise terms are generative models of some underlying phenomenon?

PS (December 7, 2012): Although it is discussed in the comments, I push my luck once more by adding a concrete example.

Consider a stochastic difference equation, $$x_{t+1} = x_t + w_t$$ where $w_t \sim \mathcal{N}(0,1)$. Then, we can write in the probabilistic terms, $$p(x_{t+1}|x_t) = \mathcal{N}(0,x_t)$$ and this is a generative model. Of course, we can write another equation, such as an observation model, and this picture completely define a discrete-time stochastic dynamical system as well as a generative model. Then I reask my question: is there a specific reference that promotes this relationship? Or is it quite trivial and obvious for researchers from both sides?

Thanks.

-
This sounds more like a matter of speculation best reserved for a discussion forum or chat room. Is there a particular problem of statistics, data analysis, or machine learning this question might help solve? –  whuber Nov 23 '12 at 16:03
I do not agree that it is a matter of speculation. Well defined motion models in Hamiltonian notation can be used for probabilistic inference problems therefore I wonder about whether these particular models can be represented in a some kind of unified framework. Actually, once this connection established in my mind, it could solve a plethora of machine learning problems. Thanks. –  Mark Nov 24 '12 at 17:19

Thanks for the reference (I can't upvote (reputation issues)). When a stochastic differential equation is converted to a difference equation, in a probabilistic framework, it immediately defines a generative model. For instance define an equation such that $$x_{t+1} = x_t + u_t + w_t$$ where $w_t$ is an arbitrary noise term with associated probability distribution. I think, this is a stochastic difference equation and also is a generative model. I wonder about, is there a specific reference which promotes this obvious connection? –  Mark Nov 24 '12 at 17:22
I am working on inference problems in dynamical systems which involve models like above mentioned model. The motivation is: since these models can be represented as a generative model, inference algorithms can be immediately applied, therefore they can use for inference problems. To make concrete, suppose simpler model, $$x_{t+1} = x_t + w_t$$ where $w_t$ is a zero mean, unit variance Gaussian random variable and $x, w \in \mathbb{R}$. Then, this model will correspond to the following generative model:$$p(x_{t+1}|x_t) = \mathcal{N}(x_t,1)$$ or $$x_{t+1}|x_t \sim \mathcal{N}(x_t,1)$$ –  Mark Nov 24 '12 at 19:19