I know a few stochastic processes that are described as 'distributions over distributions'. For example, the Dirichlet process draws discrete distributions from a default model (see e.g., p3 of these notes) .

The discrete distributions could be made continuous by assuming some kinds of kernels. E.g., if you have a discrete distribution with a probability $p_i$ associated to a variable $x_i$, you could make a continuous distribution from Gaussian kernels, $$ p(x) = \sum_i p_i \cdot g(x | \mu = x_i, \sigma^2 = 1) $$ where $g$ is a Gaussian.

Furthermore, although the discrete distributions from a Dirichlet process are a 'small' set of the continuous ones, it can be argued that they are a 'large' set in the weak or pointwise sense (see e.g., p4 of these notes). Thus, a Dirichlet process could be used to relax a parametric model in a non-parametric way.

However, are there any distributions over continuous distributions? That is, is there e.g. a stochastic process like the Dirichlet one that covers a set of continuous distributions in a strong rather than weak sense? Is there e.g. a meaningful continuum limit of the Dirichlet process that could be used for this purpose? Are there other possibilities?


closed as unclear what you're asking by whuber Aug 9 '18 at 14:31

Please clarify your specific problem or add additional details to highlight exactly what you need. As it's currently written, it’s hard to tell exactly what you're asking. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

  • $\begingroup$ Every Bayesian model concerns a "distribution of distributions," whence any Bayesian model of a continuous distribution--such as the textbook Normal distribution--will serve to answer your question. $\endgroup$ – whuber Aug 7 '18 at 11:52
  • $\begingroup$ Can you elaborate? I don’t see how a Normal distribution could be an answer to my question. $\endgroup$ – innisfree Aug 8 '18 at 1:31
  • $\begingroup$ Let $X$ be a random variable with a Normal distribution and let $Y$ be a Normal distribution of unit variance and mean $X$: That's what might be understood by a "distribution of distributions." If that's not your understanding, then please edit your question to clarify what you mean. $\endgroup$ – whuber Aug 8 '18 at 21:43
  • $\begingroup$ I see, but the example is obviously parametric and only covers the set of Gaussian distributions of unit variance. Let me clarify $\endgroup$ – innisfree Aug 8 '18 at 23:48
  • 1
    $\begingroup$ ok, I hope it is clearer now, please reopen. $\endgroup$ – innisfree Aug 10 '18 at 0:32