0
$\begingroup$

I am having a lot of difficulty understanding how to apply the algorithm to a real scenario.

The part that confuses me is that we are looking for a target distribution (the real distribution of our parameters), but we somehow know a kernel distribution that is proportional to the target distribution, since we use it to compute the acceptance ratio. How do we find/define that kernel distribution?

I would like to understand this through a concrete example. Let's say I have real-life data recorded from a system that is modelled by a PDE with 5 parameters. My goal is to create probability distributions of each of those parameters. I understand the idea of the MH-MCMC algorithm, but when it comes to computing the acceptance ratio (which requires the kernel distribution), I don't understand how we define it.

I might be missing something obvious due to my lack of statistics background.

Improve this question
$\endgroup$
5
  • $\begingroup$ you need to know the target density up to a constant, but the kernel is usually understood as the density of the proposal distribution which is completely arbitrary. $\endgroup$
    – Xi'an
    Commented Jan 24, 2021 at 21:23
  • $\begingroup$ @Xi'an What do you mean by “up to a constant”? Also, isn’t the kernel a distribution that is proportional to the target distribution, and not the proposal distribution? $\endgroup$
    – Radu
    Commented Jan 24, 2021 at 23:34
  • $\begingroup$ To understand what "up to a constant" means, consider a target f(x)=g(x)C, and you can only evaluate g(x) but not f(x) because you don't know C. Think of C as the constant that normalizes g(x) to f(x). You only need to evaluate g(x) because in the acceptance ratio of MH, you actually evaluate f(x), but it simplifies g(y)c/[g(x)c=]g(y)/g(x). $\endgroup$
    – fool
    Commented Jan 25, 2021 at 7:25
  • $\begingroup$ @user228809 Oh, evaluating g(y)/g(x) is the same as f(y)/f(x), I see! Now I’m still asking my initial question, how can we find a g function proportional to f? Considering my example above. $\endgroup$
    – Radu
    Commented Jan 25, 2021 at 8:10
  • $\begingroup$ You use MCMC to draw samples from a desired target distribution f(x)=g(x)C. For the simple case, you should know g(x) as part of your problem. For the kernel/proposal distribution, this is a choice you make (there are adapative schemes), but you can start with N(x, 1) for example. A quick googled example: stephens999.github.io/fiveMinuteStats/MH_intro.html $\endgroup$
    – fool
    Commented Jan 25, 2021 at 18:04

1 Answer 1

Reset to default
1
$\begingroup$

To compute the Metropolis-Hastings ratio for a target density $f(\cdot)$ with kernel proposal $k(\cdot|\cdot)$ one need compute $$\dfrac{f(y)k(x|y)}{f(x)k(y|x)}$$ for an arbitrary pair $(x,y)$ or find unbiased estimators $\hat f(y)$ and $\hat f(x)$ in the pseudo-marginal version of Andrieu & Roberts (2009).

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.