4
$\begingroup$

I would like to determine the MAP estimate $\hat\theta$ for a posterior distribution $P(\theta|x)$, where $x$ is data. The exact posterior is difficult to evaluate but I can approximate by $Q(\theta)$ and show that

$$ \left|\log\left[\frac{P(\theta|x)}{Q(\theta)}\right]\right|<\epsilon $$

for some small number $\epsilon$. Intuitively, this should translate to $\arg\max_\theta P(\theta|x)$ and $\arg\max_\theta Q(\theta)$ being close. But is there a formal result at the level of $P(\theta|x)$ and $Q(\theta)$ or do I need to prove that the gradients are close?

$\endgroup$
1
$\begingroup$

Unless the function is Lipschitz, there is no connection between the proximity of the functions and the proximity of the modes: take the two functions $$P(\theta|x)=\cases{1+\epsilon &\text{if }0<x<\epsilon\\1&\text{if }\epsilon<x<1}$$ and $$Q(\theta)=\cases{1+\epsilon &\text{if }1-\epsilon<x<1\\1&\text{otherwise}}$$ Those functions only differ by $\epsilon$ at most, but the modes are at opposite ends of $(0,1)$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.