Let $X$ be a random variable from $f(x; \theta)$, where $\theta =(\theta_1,\theta_2)$. I want to:

take a sample from this distribution using Metropolis Hastings algorithm and update the parameters simultaneously.

The proposal distribution is a bivariate normal distribution with mean the current value of the parameter $\theta$ and covariance matrix $\Sigma$.


Given $\theta$

Step 1: Generate $\theta^{can}$ from the proposal distribution.

Step 2: Take $\theta$ = $\theta^{can}$ with probability $\alpha(\theta,\theta^{can}) $

where $\alpha(\theta,\theta^{can})$ = $ min \left( \frac{f(\theta^{can})}{f(\theta)}∗ \frac{proposal(\theta)}{proposal(\theta^{can})}, 1\right)$

I know that

$proposal(\theta^{can}) = \frac{1}{\sqrt{(2\pi)^k|\boldsymbol\Sigma|}} \exp\left(-\frac{1}{2}({\theta^{can}}-{\theta})^T{\boldsymbol\Sigma}^{-1}({\theta^{can}}-{\theta}) \right)$.

I would like to ask if $proposal(\theta) = 0$.

  • 1
    $\begingroup$ Your description is subtly wrong; $\text{proposal}(\theta)$ should be $\text{proposal}(\theta | \theta^{can})$ and similarly for $\text{proposal}(\theta^{can})$, which should be $\text{proposal}(\theta^{can} | \theta)$. Try writing out the expressions for the two proposals and finding the ratio, cancelling out all the terms that you can. And no, $\text{proposal}(\theta) \ne 0$. $\endgroup$ – jbowman Jan 21 '14 at 20:17
  • $\begingroup$ You define "proposal" to be the proposal density. The density at $\theta$ certainly isn't zero ($e^0$ isn't $0$), though the probability of getting $\theta$ again in step 1 will be zero (this is simply a property of continuous densities). $\endgroup$ – Glen_b Jan 22 '14 at 6:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.