4
$\begingroup$

In Bayesian inference we end up with the formula:

$$ P(\mathbf{w|t,X)}= \frac{P(\mathbf{t|w,X)}P(\mathbf{w)}}{\int P(\mathbf{t|w,X}) P(\mathbf{w}) d\mathbf{w}}$$

Assume the prior $P(w)$ is a Gaussian distribution with 0 mean and $\sigma$ as standard deviation.

It is always said the integral has no closed-form solution. When the prior is a Gaussian distribution, is this related to the fact that the indefinite integral $\int e^{-x^2}dx$ can not be expressed with elementary functions?

What if I choose an ad-hoc prior distribution? Is there any case when the integral has a closed-form solution?

Improve this question
$\endgroup$
3
  • 6
    $\begingroup$ Integrals are available in closed form for special entries. In the Bayesian setting, this is essentially the case for exponential family sampling distributions and so-called conjugate priors. $\endgroup$
    – Xi'an
    Commented Oct 10, 2019 at 11:36
  • $\begingroup$ @ Xi'an is right about the use of conjugate priors making it unnecessary to evaluate the integral in the denominator of the RHS of your display. But suitable conjugate priors are not always available, so you sometimes do have to evaluate the integral. Often suitable methods of numerical integration (from almost trivial to MCMC and Metropolis-Hastings) are available. $\endgroup$
    – BruceET
    Commented Oct 12, 2019 at 4:31
  • $\begingroup$ It is also depend on your model, not only on the prior. for some models (i.e. the firs part of the integral P(t|w, X) it has closed form solution but not for any model. $\endgroup$
    – ofer-a
    Commented Mar 30, 2021 at 8:20

1 Answer 1

Reset to default
1
$\begingroup$

Yes, the marginal likelihood has a closed-form for all polynomial models of the form $\mathbf{t} = X\mathbf{w} + \boldsymbol{\varepsilon}$, where, \begin{aligned} X &= \begin{bmatrix} \mathbf{1}^T & \mathbf{(x^1)}^T & (\mathbf{x}^2)^T ... (\mathbf{x}^n)^T \end{bmatrix}\\ \boldsymbol{\varepsilon} &\sim \mathcal{N}(0, \sigma_n^2I)\\ \mathbf{w} &\sim \mathcal{N}(0,\sigma_w^2I) \end{aligned}

It is given as follows,

$$ p(t|X) \sim \mathcal{N}(0, \sigma_w^2XX^T + \sigma_n^2I) $$

I have referred to the last slide from here

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.