1
$\begingroup$

I have a model I want to fit parameters to. I know the likelihood $P(D|\theta)$. Let's say I have two parameters with prior belief, $\theta^1 \sim \mathcal{N}(\mu_2,\sigma_2^2)$, $\theta^2 \sim \mathcal{N}(\mu_1,\sigma_1^2)$. However, I know that both parameters are positive and so I bound it below by 0.

For the proposal distribution $q$, I'm using a normal centred at the previous value of $\theta^i$, with variance some scalar multiple of the initial value of the parameter (so different for each parameter). I can draw a random smaple from this proposal distribution to obtain a new sample, \theta'. (I can then repeat this for each parameter using their respective proposal distribution). However, I'm keeping in mind that the parameterds are bounded below by 0.

To calculate the acceptance ratio, I need to find

$\alpha = \min\{1,\frac{\mathcal{L}(D|\theta')q(\theta_{t-1}|\theta')P(\theta')}{\mathcal{L}(D|\theta_{t-1})q(\theta'|\theta_{t-1})P(\theta_{t-1})}\}$.

Normally, I'd be happy that I could ignore the proposal distribution here as it is symmetric. However, is this still this case given that I don't want negative parameter values? I'm a bit confused as to how I calculate $P(\theta_i)$. Does it just mean calculate $\frac{1}{\sqrt{2\pi \sigma_i^2}}e^{-\frac{(\theta^i - \mu_i)^2}{2\sigma_i^2}}$, and plug that number in? (So the value of the pdf, rather than the 'probability')? How do I incorporate the 'truncatedness' of the distribution when doing this? Or would I be better off using a gamma prior?

Example:

Would the following be correct: Let's just consider one parameter $\theta$ with prior belief $\theta \sim \mathcal{N^+}(0.16,0.06^2)$. Let's say $\theta_{t-1} = 0.14$. I draw a new proposal from the proposal distribution $\mathcal{N^{+}}(0.14, 0.05^2)$, which gives me $\theta' = 0.15$, say. Then the exponential part of the proposal pdf will be the same for both the numerator and denominator of the acceptance ratio, so I only need to deal with the normalising constant. So the proposal ratio part of alpha just becomes $\frac{\Phi(\theta_{t-1}/\sigma)}{\Phi(\theta' /\sigma)} = \frac{\Phi(0.14/0.05)}{\Phi(0.15/0.05)}$

Does this then mean that overall, my acceptance ratio becomes

$\alpha = \min\left\{1,\frac{\mathcal{L}(D|\theta')\Phi(0.14/0.05)e^{-\frac{(\theta'-0.16)^2}{2(0.06)^2}}}{\mathcal{L}(D|\theta_{t-1})\Phi(0.15/0.05)e^{-\frac{(\theta_{t-1}-0.16)^2}{2(0.06)^2}}}\right\}$.

$\endgroup$
8
  • $\begingroup$ Could you clarify the "keeping in mind that the parameterds are bounded below by 0"? If you use a truncated Normal as a proposal in MCMC it is no longer symmetric and you need to compute its normalising constant in the Metropolis acceptance probability. $\endgroup$
    – Xi'an
    May 7, 2020 at 12:03
  • $\begingroup$ @Xi'an I've added an example in my post. Is that correct, or have I misunderstood? $\endgroup$
    – user112495
    May 7, 2020 at 18:03
  • $\begingroup$ You could also work with log-transformed parameters. $\endgroup$
    – Björn
    May 7, 2020 at 18:12
  • $\begingroup$ @Xi'an I thought I had? I've included the rest of the normalising constants (at least, what I thought they would be), but don't they cancel out, as the bits in the square roots only include the standard deviation which will cancel out from top to bottom? $\endgroup$
    – user112495
    May 8, 2020 at 8:50
  • 1
    $\begingroup$ yes $\alpha$ is now correct. $\endgroup$
    – Xi'an
    May 8, 2020 at 12:16

0

Browse other questions tagged or ask your own question.