4
$\begingroup$

I'm confused in the following situation:

I want to sample by writing code (Java) from the following distribution that is characterized by the mean vectors and covariance matrices:

$$ p\left ( \mathbf{x} \right ) = \frac{w_{1}}{2\pi \sqrt{\left | \mathbf{C}_{1} \right |}}e^{-\frac{1}{2}\left ( \mathbf{x}-\mu _{1} \right )^{\top }C_{1}^{-1}\left ( \mathbf{x}-\mu _{1} \right )} + \frac{w_{2}}{2\pi \sqrt{\left | \mathbf{C}_{2} \right |}}e^{-\frac{1}{2}\left ( \mathbf{x}-\mu _{2} \right )^{\top }C_{2}^{-1}\left ( \mathbf{x}-\mu _{2} \right )} $$ where for $$ i = 1, 2, \mu_{i} \hspace{35pt} \mathrm{and}\hspace{35pt} \mathbf{C}_{i}$$ are respectively the mean vectors and the covariance matrices. $$ (\mathbf{x}-\mu _{i})^{\top } $$ is the row vector corresponding with the column vector $$ (\mathbf{x}-\mu _{i}) $$

We assume that the first and second mean are the column vectors $$ \boldsymbol{\mu }_{1} = (0,0)^{\top } \enspace \mathrm{and }\enspace \boldsymbol{\mu }_{1} = (d,0)^{\top } $$ and the first and second covariance matrices are $$ \mathbf{C}_{1} = \begin{pmatrix} \sigma _{1} & 0\\ 0 & \sigma _{2} \end{pmatrix} \enspace \mathrm{and }\enspace \mathbf{C}_{2} = \begin{pmatrix} a & b\\ b & d \end{pmatrix} \enspace \mathrm{respectively}. $$

Furthermore, $$ w_{1} = 0.7 ,\thinspace w_{2} = 0.3 $$ and $$ \mu _{1} = \binom{4}{5}, \mu _{2} = \binom{0.7}{3.5}, \mathbf{C}_{1} = \begin{pmatrix} 1.0 & 0.7\\ 0.7 & 0.1 \end{pmatrix},$$ and $$ \mathbf{C}_{2} = \begin{pmatrix} 1.0 & -0.7\\ -0.7 & 1.0 \end{pmatrix} $$

Since I'm using symmetric proposal distributions in the Metropolis-Hastings algorithm, the acceptance probability reduces to

where $$ \mathbf{x} \enspace \mathrm{and } \enspace \mathbf{x}^{*} $$ are respectively the current state and the proposed new state accepted with probability $$ A(\mathbf{x},\mathbf{x}^{*}) $$

I want to sample for both the Gaussian distribution and the uniform distribution. The Gaussian distribution is $$ q(\mathbf{x^{*}},\mathbf{x}) = \frac{1}{2\pi\sqrt{\left | \mathbf{C} \right |}}e^{-\frac{1}{2}(\mathbf{x^{*}-x})^{\top }\mathbf{C^{-1}}(\mathbf{x^{*}}-\mathbf{x})} $$ where $$ \mathbf{C} = \begin{pmatrix} \sigma & 0\\ 0& \sigma \end{pmatrix} $$ The uniform distribution is over the square $$ [x_{1} - r, x_{1} + r] \times [x_{2} - r, x_{2} + r]$$ $$ q(\mathbf{x^{*}},\mathbf{x}) = \begin{Bmatrix} \frac{1}{4r^{2}} & \mathrm{for} \enspace x_{1}^{*} \in [x_{1} - r, x_{1} + r]\enspace \mathrm{and } \enspace x_{2}^{*} \in [x_{2} - r, x_{2} + r]\\ 0 & \mathrm{otherwise} \end{Bmatrix} $$

I'm stuck in how to implement this in order to be able to do the sampling.

$\endgroup$
1
$\begingroup$

Let $x^{(i)}$ be the current value in the chain. Using the normal proposal distribution as an example, sample $x^* \sim N(x^{(i)}, C)$ and set $x^{(i+1)} = x^*$ with probability $A(x^{(i)},x^*)$ and otherwise set $x^{(i+1)} = x^{(i)}$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.