Do Stochastic Processes such as the Gaussian Process/Dirichlet Process have densities? If not, how can Bayes rule be applied to them? The Dirichlet Pocess and Gaussian Process are often referred to as "distributions over functions" or "distributions over distributions".  In that case, can I meaningfully talk about the density of a function under a GP? That is, do the Gaussian Process or Dirichlet Process have some notion of a probability density?
If it does not, how can we use Bayes' rule to go from prior to posterior, if the notion of the prior probability of a function is not well defined?  Do things such as MAP or EAP estimates exist in the Bayesian Nonparametric world?  Thanks a lot.
 A: A "density" or "likelihood" relates to the Radon-Nikodym theorem in
measure theory. As noted by @Xi'an, when you consider a finite set of
so-called partial observations of a stochastic process, the
likelihood corresponds to the usual notion of derivative w.r.t. the Lebesgue
measure. For instance, the likelihood of a Gaussian process observed at
a known finite set of indices is that of a Gaussian random vector with
its mean an covariance deduced from that of the process, which can
both take parameterized forms.
In the idealized case where an infinite number of observations is
available from a stochastic process, the probability measure is on an
infinite-dimensional space, for instance a space of continuous
functions if the stochastic process has continuous paths. But nothing
exists like a Lebesgue measure on an infinite-dimensional space, hence
there is no straightforward definition of the likelihood.
For Gaussian processes there are some cases where we can define a
likelihood by using the notion of equivalence of Gaussian measures. An
important example is provided by Girsanov's theorem, which is widely used
in financial math. This defines the likelihood of an Itô diffusion
$Y_t$ as the derivative w.r.t the probability distribution of a
standard Wiener process $B_t$ defined for $t \geq 0$. A neat math
exposition is found in the book by Bernt
Øksendal. The
(upcoming) book by Särkkä and Solin
provides a more intuitive presentation which will help practitioners.
A brilliant math exposition on Analysis and Probability on
Infinite-Dimensional Spaces by Nate
Elderedge is available.
Note that the likelihood of a stochastic process that would be
completely observed is sometimes called infill likelihood by
statisticians.
