3
$\begingroup$

Consider the (hierarchical) Bayesian inference problem with two unknowns $(x,\theta)$ and data $y$. I'm using a very simple ("independence"?) approximation $$ p(x,\theta|y) \approx p(x|\theta_\star,y) \, p(\theta|y) \, ,$$ where $\theta_\star$ is the mode of $p(\theta|y)$ -- using other point estimates for $\theta_\star$ is also an option.

Question: does this type of approximation have a name? Note that

  • Unlike "Laplace approximation", I do not want to integrate out $\theta$.
  • Unlike "Variational Bayes" I'm not approximating $p(\theta|y)$ and $\theta_\star$ is obtained in a much simpler fashion.

I'm asking so that I may look up some literature on it. For instance, how might it compare to mean-field approximations?

Edit: I think it might be appropriate to call it "empirical Bayes without marginalization". Still, any insights/opinions would be welcome.

Improve this question
$\endgroup$
2
  • $\begingroup$ If $\theta^{\ast}$ is the mode of $p(\theta|y)$, then I think what you are doing on the right hand side of the equation is some form of type-II Maximum Likelihood. I.e., you take $\theta$ as some hyperparameter influencing both $x$ and $y$, and rather than integrating out the parameter uncertainty about $\theta$ encoded in a potential prior distribution, you use the ML estimate. That being said, I find the approximation itself a little strange, since it basically says $P(x,\theta|y) \approx P(x|\theta = \theta^{\ast}, y)$. Are you sure that is reasonable to assume? $\endgroup$
    – Jeremias K
    Commented Nov 27, 2017 at 13:43
  • $\begingroup$ @JeremiasK so you agree with the edit: it's Empirical Bayes (without marginalization)? As for its validity, it is of course questionable, but not more so than EB, afaik. Also, you seem to be missing $p(\theta|y)$. $\endgroup$
    – Patrick
    Commented Nov 27, 2017 at 17:41

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.