# Bayesian inference with the wrong distribution

When an observation $x$ is generated by $P(x|\theta)$ for a parameter $\theta$ the Bayesian optimal estimator for the value of $\theta$ is $\hat\theta_{BEST}=\mathbb{E}[\theta|x]=\frac{1}{P(x)}\int d\theta P(\theta)P(x|\theta)$.

Now assume that instead of using $P(x|\theta)$ in the above formula we have $Q(x|\theta)$ which is different from $P$ but have the same mean and variance for all $\theta$; that is $\hat\theta_{EST}=\frac{1}{Q(x)}\int d\theta P(\theta)Q(x|\theta)$ and $\mathbb{E}[Q(x|\theta)]=\mathbb{E}[P(x|\theta)]$ and $\mathbb{V}[Q(x|\theta)]=\mathbb{V}[P(x|\theta)]$ for all $\theta$.

Are there any known boundaries on $MSE[\hat\theta_{EST}]-MSE[\hat\theta_{BEST}]$?

What if we further assume Gaussianity of $Q$?

Thanks!

• Related but not the same as math.stackexchange.com/questions/712040/… – Henry Mar 14 '14 at 19:42
• Esp. in the case of Gaussian $Q$, you're doing a saddle point approximation, so searching on bounds for the errors in that might be useful. – Dave Mar 25 '14 at 21:16
• @Dave, I didn't understood your idea: do you suggest to calculate $\hat{\theta}_{EST}$ using a saddle-point approximation? This is exactly MAP (maximal a-posteriori), no? What kind of bounds this may provide? – Uri Cohen Mar 26 '14 at 12:52