2
$\begingroup$

Suppose I have a strictly positive parameter $\sigma$ and I need to estimate it using the random walk Metropolis-Hasting algorithm.

I know that I can do a parameter transform, i.e., $\beta=log(\sigma)$ and estimate $\beta$ instead. It is more convenient for me to assume that the prior for $\beta$ follows a normal distribution, e.g., $\beta ~ N(\alpha,\lambda)$. ($\alpha$ and $\lambda$ are actually parameters from other parts of my model). Then the log likelihood for the prior is,

$logprior= N(\beta,\alpha,\lambda,loglike=true)$

My confusion is whether I should add the Jacobian term or not. Suppose I still need to add the Jacobian term, should I just make the log of the prior look like this?

$logprior= N(\beta,\alpha,\lambda,loglike=true) + \beta$

Is $\beta$ the correct Jacobian term? I deduced it from this answer, but I don't know if I understand it correctly.

$\endgroup$
1
  • $\begingroup$ If you’re imposing a prior on the transformed parameter, you don’t need the Jacobian $\endgroup$
    – Daeyoung
    Mar 26, 2022 at 2:39

1 Answer 1

2
$\begingroup$

The Jacobian is used when you convert back and forth between a distribution for $\sigma$ and a distribution for $\beta$. This is true of the conversion whether you are talking about the prior or posterior distribution. You don't need to add it when you are dealing with a posterior computation for a given parameter that is not changing. So, if you are making an inference about $\beta$, you can derive its posterior distribution without any use of the Jacobian, as:

$$\pi(\beta|\mathbf{x}) \propto L_\mathbf{x}(\beta) \cdot \pi(\beta).$$

Similarly, if you are making an inference about $\sigma$, you can derive its posterior distribution without any use of the Jacobian, as:

$$\pi'(\sigma|\mathbf{x}) \propto L_\mathbf{x}'(\sigma) \cdot \pi'(\sigma).$$

(I'm using the primes here to reflect the fact that these sets of functions are different from each other, since they are for different paramters.) Since $\beta = \log \sigma$ these functions are related by:

$$\begin{align} L_\mathbf{x}'(\sigma) &\overset{\sigma}{\propto} L_\mathbf{x}(\log \sigma), \\[12pt] \pi'(\sigma) &\overset{\sigma}{\propto} \pi(\log \sigma) \cdot \frac{1}{\sigma}, \\[12pt] \pi'(\sigma|\mathbf{x}) &\overset{\sigma}{\propto} \pi'(\sigma|\mathbf{x}) \cdot \frac{1}{\sigma}, \\[12pt] L_\mathbf{x}(\beta) &\overset{\beta}{\propto} L_\mathbf{x}'(e^\beta), \\[12pt] \pi(\beta) &\overset{\beta}{\propto} \pi'(\exp \beta) \cdot e^\beta, \\[12pt] \pi(\beta|\mathbf{x}) &\overset{\beta}{\propto} \pi'(e^\beta|\mathbf{x}) \cdot e^\beta. \\[12pt] \end{align}$$

As you can see, the Jacobian enters into these relationships when we convert either the prior or posterior distribution for one parameter to the other.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.