I know that when the prior is conjugate with posterior then one can get an analytical representation for the posterior distribution, but what if these two are not to be conjugate? For example, I would like to use Lasso Bayesian, in which likelihood is the normal distribution and the prior is the Laplace distribution. Then I would like to know, do I still get an approximate sparse solution for the ordinary least square problem?
2
-
1$\begingroup$ This is how something like Markov chain Monte Carlo (MCMC) comes up in order to do approximations. Conjugates are nice because no such computation is required, but they could be terrible priors. $\endgroup$– DaveAug 27, 2021 at 19:49
-
$\begingroup$ Hey @Dave, thank you, so if I would like to approach Bayesian linear approximation by using Laplace prior, can I use MCMC in this way? $\endgroup$– RazAug 27, 2021 at 21:42
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
3
In most cases we don't have priors that would be conjugate to the likelihood function and result in a closed-form solution. In such a case, we are left with approximate solutions, such as grid approximation for simple problems, and for more realistic ones Laplace approximation, Markov Chain Monte Carlo (MCMC) (see mcmc) sampling, variational inference (see variational-bayes), and others. If you care only about the point estimate of the mode, you can use optimization to obtain the maximum a posterior (MAP) estimate. Those methods have different pros and cons, for example, Laplace approximation and variational inference are fast but less accurate, while MCMC is slow and computationally demanding, but more precise if you draw large enough samples.
-
$\begingroup$ Thank you @Tim, so if I would like to approach Bayesian linear approximation by using Laplace prior, can I use MCMC in this way? $\endgroup$– RazAug 27, 2021 at 21:41
-
1$\begingroup$ @Raz MCMC can be used to sample from any distribution known up to a constant. $\endgroup$– Tim ♦Aug 27, 2021 at 21:44
-