# Proposal in MCMC lives in bigger space than parameter space. Which transformations should I choose?

I'm using a MCMC algorithm. The proposal is, due to lack of information on my part, a multivariate T-Student distribution, i.e. $$\theta \sim \mathcal{MT}(\mu, \Sigma)$$. However, some of the components of $$\theta$$ are restricted to a specific interval. To make sure that most of the draws from the proposal do not get rejected due to a acceptance probability equal to 0, I'm thinking of using some transformations.

For example, for the components which take values in $$]0,1[$$, I'm thinking of using a logit transformation. For $$]0,+\infty[$$ a $$\log$$ transformation. For $$]-1,1[$$, I will translate to $$]-\pi/2,\pi/2[$$ and apply $$\tan$$ .

What other transformations maybe used? Is this a good policy? What precautions should I take to choose good transformations?

• Make sure you do not forget the Jacobians. – Xi'an Dec 2 '18 at 16:52
• @Xi'an the model I'm using makes it too hard to compute the jacobian. By hand, I simply do not know how to. By PC, I've tried defining the likelihood in Mathematica, then derive (1st derivative) the corresponding expression. with a sample size of 10, and my PC runs out of memory. Maybe, I'm missing something. To find the 1st derivative, I had to use some formulas involving derivation of a scalar function w.r.t. a matrix, and the kronecker product, and vec operator. How does one usually find the Jacobian? Sorry for this last question. ;) – An old man in the sea. Dec 2 '18 at 17:17
• Do you plan on having non-zero off-diagonal elements in $\Sigma$? – Robin Ryder Dec 2 '18 at 17:18
• @RobinRyder at the moment, I don't think so. However, I would like to. – An old man in the sea. Dec 2 '18 at 17:19
• @Xi'an in my previous comment, I mixed jacobian with hessian. Sorry. – An old man in the sea. Dec 2 '18 at 17:40

## 1 Answer

It is perfectly valid to reparametrize your model before implementing MCMC. Two caveats, as mentioned in the comments: (1) you need to calculate the Jacobian of the change of the variable; (2) depending on the problem, this can make it more difficult to think about the correlation between the parameters. Depending on the problem, it can be easier or harder to find a proposal that allows the chain to mix well.

However, since you are thinking of updating the components independently, another option is to forgo the $$t$$ distribution for certain parameters. In particular, your transition kernel can update only one parameter at each step, instead of updating all parameters at once. You can then choose a proposition which won't (at all/too often) propose values outside of the interval of support.

Edited to add: this answer by jbowman is very relevant and more detailed.

• Just to be sure, the only place where I have to calculate the jacobian is for the prior, right? In the likelihood, nothing changes. – An old man in the sea. Dec 3 '18 at 19:39
• @Anoldmaninthesea. You compute the jacobian for the change of variables only once, and you then write everything in terms of the new parameterization. – Robin Ryder Dec 3 '18 at 19:59
• Robin, I have a doubt. If some of the parameters are reparametrized for the proposal, but the prior is given with respect to the reparametrization(not the original), do I still need to consider these parameters when computing the jacobian? – An old man in the sea. Feb 24 at 16:57
• @Anoldmaninthesea. Are you still using the original somewhere? Or are all of {prior, likelihood, proposal} defined with the reparametrization? – Robin Ryder Feb 25 at 8:00
• In that case, if I understand correctly, you no longer need a Jacobian, since you can effectively express everything in terms of the new parameters. – Robin Ryder Mar 1 at 12:50