1
$\begingroup$

I'm using a MCMC algorithm. The proposal is, due to lack of information on my part, a multivariate T-Student distribution, i.e. $\theta \sim \mathcal{MT}(\mu, \Sigma)$. However, some of the components of $\theta$ are restricted to a specific interval. To make sure that most of the draws from the proposal do not get rejected due to a acceptance probability equal to 0, I'm thinking of using some transformations.

For example, for the components which take values in $]0,1[$, I'm thinking of using a logit transformation. For $]0,+\infty[$ a $\log$ transformation. For $]-1,1[$, I will translate to $]-\pi/2,\pi/2[$ and apply $\tan$ .

What other transformations maybe used? Is this a good policy? What precautions should I take to choose good transformations?

Improve this question
$\endgroup$
7
  • 2
    $\begingroup$ Make sure you do not forget the Jacobians. $\endgroup$
    – Xi'an
    Commented Dec 2, 2018 at 16:52
  • $\begingroup$ @Xi'an the model I'm using makes it too hard to compute the jacobian. By hand, I simply do not know how to. By PC, I've tried defining the likelihood in Mathematica, then derive (1st derivative) the corresponding expression. with a sample size of 10, and my PC runs out of memory. Maybe, I'm missing something. To find the 1st derivative, I had to use some formulas involving derivation of a scalar function w.r.t. a matrix, and the kronecker product, and vec operator. How does one usually find the Jacobian? Sorry for this last question. ;) $\endgroup$
    – An old man in the sea.
    Commented Dec 2, 2018 at 17:17
  • $\begingroup$ Do you plan on having non-zero off-diagonal elements in $\Sigma$? $\endgroup$
    – Robin Ryder
    Commented Dec 2, 2018 at 17:18
  • $\begingroup$ @RobinRyder at the moment, I don't think so. However, I would like to. $\endgroup$
    – An old man in the sea.
    Commented Dec 2, 2018 at 17:19
  • $\begingroup$ @Xi'an in my previous comment, I mixed jacobian with hessian. Sorry. $\endgroup$
    – An old man in the sea.
    Commented Dec 2, 2018 at 17:40

1 Answer 1

Reset to default
2
$\begingroup$

It is perfectly valid to reparametrize your model before implementing MCMC. Two caveats, as mentioned in the comments: (1) you need to calculate the Jacobian of the change of the variable; (2) depending on the problem, this can make it more difficult to think about the correlation between the parameters. Depending on the problem, it can be easier or harder to find a proposal that allows the chain to mix well.

However, since you are thinking of updating the components independently, another option is to forgo the $t$ distribution for certain parameters. In particular, your transition kernel can update only one parameter at each step, instead of updating all parameters at once. You can then choose a proposition which won't (at all/too often) propose values outside of the interval of support.

Edited to add: this answer by jbowman is very relevant and more detailed.

Improve this answer
$\endgroup$
6
  • $\begingroup$ Just to be sure, the only place where I have to calculate the jacobian is for the prior, right? In the likelihood, nothing changes. $\endgroup$
    – An old man in the sea.
    Commented Dec 3, 2018 at 19:39
  • $\begingroup$ @Anoldmaninthesea. You compute the jacobian for the change of variables only once, and you then write everything in terms of the new parameterization. $\endgroup$
    – Robin Ryder
    Commented Dec 3, 2018 at 19:59
  • $\begingroup$ Robin, I have a doubt. If some of the parameters are reparametrized for the proposal, but the prior is given with respect to the reparametrization(not the original), do I still need to consider these parameters when computing the jacobian? $\endgroup$
    – An old man in the sea.
    Commented Feb 24, 2019 at 16:57
  • $\begingroup$ @Anoldmaninthesea. Are you still using the original somewhere? Or are all of {prior, likelihood, proposal} defined with the reparametrization? $\endgroup$
    – Robin Ryder
    Commented Feb 25, 2019 at 8:00
  • 1
    $\begingroup$ In that case, if I understand correctly, you no longer need a Jacobian, since you can effectively express everything in terms of the new parameters. $\endgroup$
    – Robin Ryder
    Commented Mar 1, 2019 at 12:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.