MCMC proposition from an underlying distribution

Question

I'm a newbie when it comes to MCMC. I'm working with a program that is used to fit a number of parameters given some data. The documentation is leaving me a bit confused, I'm hoping someone here could help put me on the right track.

For example, say I am fitting 3 parameters, $\alpha$, $\beta$, and $\gamma$. I have some knowledge of the underlying distributions of $\alpha$ and $\beta$, but not $\gamma$.

This program allows me to input my prior knowledge of these distributions by indicating the underlying distribution of each parameter. For example, I can tell the program:

$\alpha$~$N(0,1)$

$\beta$~$exponential(3)$

$\gamma$~$uniform(0,\infty)$

After writing up a control file indicating these parameters and distributions, I need to set up the 'MCMC' run. There are a number of different moves I can choose from, including (copied from the manual):

log uniform proposal: A Metropolis-Hastings move for continuous scalar variables that are constrained to be positive only (or negative only). If x is the current state then the proposed state is $xe^U$, where $U$ is simulated from a continuous uniform distribution with range 0 ± half-width. This move is equivalent to the uniform proposal on the logarithmic scale.

logit uniform proposal: A Metropolis-Hastings move for continuous scalar variables that are constrained between 0 and 1. If x is the current state then the proposed state is $\frac{1}{1+e^{-log{\left(\frac{x}{1-x}\right)-U}}}$ where $U$ is simulated from a continuous uniform distribution with range 0 ± half-width. This move is equivalent to the uniform proposal on the logit scale.

uniform proposal: A Metropolis-Hastings move for continuous scalar variables. If $x$ is the current state then the proposed state is $x$ + $U$, where $U$ is simulated from a continuous uniform distribution with range 0 ± half-width.

Now I'm confused by this, because it appears the moves are determined by the MCMC proposals, not the underlying distributions that I previously defined.

I'm hoping someone can help me understand how (if) the underlying distributions inform the proposed jumps in the MCMC.

Thanks!

Answer 1

Perhaps you are confusing the prior distribution with the proposal distribution.

The prior distribution is part of the posterior distribution. In particular, the posterior distribution is proportional to the product of the likelihood $p(y|\theta)$ and the prior distribution $p(\theta)$: $$ p(\theta|y) \propto p(y|\theta)\,p(\theta) , $$ where $y$ denotes the data and $\theta$ is the unknown parameter (or parameters). In your case $\theta = (\alpha,\beta,\gamma)$. As you say, you have some prior information. In particular, $\alpha \sim \textsf{N}(0,1)$ and $\beta \sim \textsf{Exp}(3)$. You also have some prior information about $\gamma$, namely $\gamma > 0$. Beyond that sign restriction, you wish to use an improper prior $p(\gamma) \propto 1_{[0,\infty]}(\gamma)$.

The proposal distribution plays a role in the Metropolis-Hastings MCMC sampler that is used to make draws from the posterior distribution. Let $q(\theta'|\theta)$ denote the density of the proposal distribution for $\theta'$ given $\theta$. Let $\theta^{(r)}$ denote the current state of the Markov chain. Then $$ \theta^{(r+1)} = \begin{cases} \theta' & R \ge u \\ \theta^{(r-1)} & \text{otherwise} \end{cases} $$ where $u \sim \textsf{Uniform}(0,1)$ and $$ R = \frac{p(y|\theta')\,p(\theta')}{p(y|\theta^{(r)})\,p(\theta^{(r)})} \times \frac{q(\theta^{(r)}|\theta')}{q(\theta'|\theta^{(r)})} . $$

It appears the software allows you to choose different proposal distributions depending on the prior bounds of the parameters. Since $\alpha \in (-\infty, \infty)$ you would choose the uniform proposal for it. Your other two parameters are constrained to be positive, so you would choose the log-uniform proposal for them.

(I am not endorsing these proposal distributions; I am only trying to help you understand what I think the user guide is trying to tell you.)