3
$\begingroup$

In Bayesian Data Analysis, PDF freely available, section 4.1 (page 84, bottom) there is a comment saying:

If we had instead constructed the normal approximation in terms of $p(\mu, \sigma^2)$, the second derivative matrix would be multiplied by the Jacobian of the transformation from $\log\sigma$ to $\sigma^2$ and the mode would change slightly, to $\tilde{\sigma}^2 = \frac{n}{n+2}\hat{\sigma}^2$.

My question is how do we compute the Jacobian from $\log\sigma$ to $\sigma^2$? I can differentiate one function with respect to the other but I can't reconcile the stated change in mode (which makes me think I'm mistaken).

Thanks in advance

$\endgroup$
4
$\begingroup$

For the parameter transform$$\eta\longmapsto\sigma^2=\exp\{2\eta\}$$the Jacobian is $$\frac{\text d\sigma^2}{\text d\eta}=2\exp\{2\eta\}=2\sigma^2$$and the posterior changes from $p(\mu,\eta)$ into$$p^\prime(\mu,\sigma^2)=p(\mu,\log(\sigma^2)/2)\times \frac{\text d\eta}{\text d\sigma^2} = p(\mu,\log(\sigma^2)/2) \frac{1}{2\sigma^2}$$which modifies the location of the mode: $$\arg\max_{\mu,\sigma^2}p(\mu,\log(\sigma^2)/2)\ne \arg\max_{\mu,\sigma^2}p(\mu,\log(\sigma^2)/2)\frac{1}{2\sigma^2}$$ Namely, $$\hat\sigma^2 = \arg\max_{\sigma^2} \left\{-n\log \sigma-\frac{n \hat\sigma^2}{2\sigma^2}\right\}$$ versus $${\tilde\sigma}^2 = \arg\max_{\sigma^2}\left\{-n\log \sigma-\frac{n \hat\sigma^2}{2\sigma^2}\underbrace{-\log \sigma^2}_{+\log \frac{\text d\eta}{\text d\sigma^2}}\right\}=\frac{n}{n+2}\hat\sigma^2$$

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Thank you Xi'an - this is very helpful. Minor follow-up: when I said I can't reconcile the stated change in mode I meant the specific stated change in the book i.e. to $\frac{n}{n+2}\hat{\sigma}^2$. $\endgroup$ – feyninggreatness Sep 27 at 14:57
  • $\begingroup$ Wonderful, thank you. Final check: is there a typo in the Jacobian, it should be $\frac{d \eta}{d \sigma^2}$? $\endgroup$ – feyninggreatness Sep 28 at 13:57
  • 1
    $\begingroup$ Both are. The Jacobian of one transform $\eta\mapsto\sigma^2$ is the inverse of the reverse transform $\sigma^2\mapsto\eta$. (Typo corrected.) $\endgroup$ – Xi'an Sep 28 at 16:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.