# Random walk with bivariate normal distribution

Let $X$ be a random variable from $f(x; \theta)$, where $\theta =(\theta_1,\theta_2)$. I want to:

take a sample from this distribution using Metropolis Hastings algorithm and update the parameters simultaneously.

The proposal distribution is a bivariate normal distribution with mean the current value of the parameter $\theta$ and covariance matrix $\Sigma$.

Steps:

Given $\theta$

Step 1: Generate $\theta^{can}$ from the proposal distribution.

Step 2: Take $\theta$ = $\theta^{can}$ with probability $\alpha(\theta,\theta^{can})$

where $\alpha(\theta,\theta^{can})$ = $min \left( \frac{f(\theta^{can})}{f(\theta)}∗ \frac{proposal(\theta)}{proposal(\theta^{can})}, 1\right)$

I know that

$proposal(\theta^{can}) = \frac{1}{\sqrt{(2\pi)^k|\boldsymbol\Sigma|}} \exp\left(-\frac{1}{2}({\theta^{can}}-{\theta})^T{\boldsymbol\Sigma}^{-1}({\theta^{can}}-{\theta}) \right)$.

I would like to ask if $proposal(\theta) = 0$.

• Your description is subtly wrong; $\text{proposal}(\theta)$ should be $\text{proposal}(\theta | \theta^{can})$ and similarly for $\text{proposal}(\theta^{can})$, which should be $\text{proposal}(\theta^{can} | \theta)$. Try writing out the expressions for the two proposals and finding the ratio, cancelling out all the terms that you can. And no, $\text{proposal}(\theta) \ne 0$. – jbowman Jan 21 '14 at 20:17
• You define "proposal" to be the proposal density. The density at $\theta$ certainly isn't zero ($e^0$ isn't $0$), though the probability of getting $\theta$ again in step 1 will be zero (this is simply a property of continuous densities). – Glen_b Jan 22 '14 at 6:46