2
$\begingroup$

I know that when the prior is conjugate with posterior then one can get an analytical representation for the posterior distribution, but what if these two are not to be conjugate? For example, I would like to use Lasso Bayesian, in which likelihood is the normal distribution and the prior is the Laplace distribution. Then I would like to know, do I still get an approximate sparse solution for the ordinary least square problem?

$\endgroup$
2
  • 1
    $\begingroup$ This is how something like Markov chain Monte Carlo (MCMC) comes up in order to do approximations. Conjugates are nice because no such computation is required, but they could be terrible priors. $\endgroup$
    – Dave
    Aug 27, 2021 at 19:49
  • $\begingroup$ Hey @Dave, thank you, so if I would like to approach Bayesian linear approximation by using Laplace prior, can I use MCMC in this way? $\endgroup$
    – Raz
    Aug 27, 2021 at 21:42

1 Answer 1

6
$\begingroup$

In most cases we don't have priors that would be conjugate to the likelihood function and result in a closed-form solution. In such a case, we are left with approximate solutions, such as grid approximation for simple problems, and for more realistic ones Laplace approximation, Markov Chain Monte Carlo (MCMC) (see ) sampling, variational inference (see ), and others. If you care only about the point estimate of the mode, you can use optimization to obtain the maximum a posterior (MAP) estimate. Those methods have different pros and cons, for example, Laplace approximation and variational inference are fast but less accurate, while MCMC is slow and computationally demanding, but more precise if you draw large enough samples.

$\endgroup$
3
  • $\begingroup$ Thank you @Tim, so if I would like to approach Bayesian linear approximation by using Laplace prior, can I use MCMC in this way? $\endgroup$
    – Raz
    Aug 27, 2021 at 21:41
  • 1
    $\begingroup$ @Raz MCMC can be used to sample from any distribution known up to a constant. $\endgroup$
    – Tim
    Aug 27, 2021 at 21:44
  • $\begingroup$ Perfect, thank you. $\endgroup$
    – Raz
    Aug 27, 2021 at 21:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.