# Multi parameter Metropolis-Hastings

I need to formulate a multi parameter Metropolis-Hastings algorithm.

My question is related to how to define the condition to accept or not the candidate value.

In my problem (it is a curve fitting) I have 5 parameters theta=(a1,a2,a3,a4,a5), some of them are associated to a informative Prior distribution, others to a non-informative Prior distribution.

PRIOR DISTRIBUTIONS:

• a1 follows a Log-Normal(mu1,sigma1)
• a2 follows a Log-Normal(mu2,sigma2)
• a3 follows a Uniform(lower3,upper3)
• a4 follows a Uniform(lower4,upper4)
• a5 follows a Uniform(lower5,upper5)

Moreover I calculate the likelihood estimation for the parameters, lets call it Like(theta).

To decide if to accept or not the candidate value at time t, I have in mind three options, but I don't know which one is correct, because I could find only algorithms with 1 parameter:

• define the ratio R for each parameter:

$R_i=\frac{Like(theta)_t Prior(a_i)_t}{Like(theta)_{t-1} Prior(a_i)_{t-1}}$

and accept the vector theta at time t only if ALL the $R_i$ are higher than 1 or than a random number between 0 and 1.

• define the ratio R for each parameter:

$R_i=\frac{Like(theta)_t Prior(a_i)_t}{Like(theta)_{t-1} Prior(a_i)_{t-1}}$

and accept the parameter $a_i$ at time t if the $R_i$ are higher than 1 or than a random number between 0 and 1.

• define the ratio R as:

$R=\frac{Like(theta)_t Prior(a_1)_t Prior(a_2)_t Prior(a_3)_t Prior(a_4)_t Prior(a_5)_t}{Like(theta)_{t-1} Prior(a_1)_{t-1} Prior(a_2)_{t-1} Prior(a_3)_{t-1} Prior(a_4)_{t-1} Prior(a_5)_{t-1}}$

and accept the the vector theta at time t only if the $R$ is higher than 1 or than a random number between 0 and 1.

Can somebody give me an answer or a reference in which a case like this has been developed?

Thanks

You actually have a single joint prior, which is a function of the parameter vector $\theta = [a_1, ..., a_d]$. If the parameters are treated independently, the prior factorizes into a product of the 'individual priors' that you mentioned. That is:

$$p(\theta) = \prod_{i = 1}^d p(a_i)$$

Classic Metropolis-Hastings looks the same whether you have a single parameter or multiple parameters; in the multi-parameter case, you just consider the parameter vector as a single object.

Let $\theta_t$ be the current parameter vector (at step $t$), $\theta'$ be a new candidate parameter vector drawn from the proposal distribution, and $D$ be the data. Calculate the ratio:

$$R_t = \frac{p(D \mid \theta') p(\theta')}{p(D \mid \theta_t) p(\theta_t)}$$

Accept $\theta'$ if $R_t \ge 1$, otherwise accept it with probability $R_t$.

Classic Metropolis-Hastings can be slow to converge in high dimensions. If this is a problem, more advanced techniques like Hamiltonian Monte Carlo can be used.

• Thank you! Just a question, in case the Priors are not independent but correlated, how can I calculate $p(\theta)$ in five dimensions? – D.Leo Jul 20 '16 at 9:22
• It's the prior, so you get to decide what it is. Presumably you'd have an expression for it that you can just evaluate (i.e. a function of the individual parameters). – user20160 Jul 20 '16 at 9:31
• the priors are derived from inference on other data sets. – D.Leo Jul 20 '16 at 9:33
• If you want the prior to express dependence between the parameters, then you'd need a function that depends on multiple parameters, not just a set of marginal distributions for each separate parameter. So, you'd either have to infer this joint structure from your other data sets, or posit some kind of relationship yourself. – user20160 Jul 20 '16 at 9:54