3
$\begingroup$

In Bayesian Data Analysis, PDF freely available, section 4.1 (page 84, bottom) there is a comment saying:

If we had instead constructed the normal approximation in terms of $p(\mu, \sigma^2)$, the second derivative matrix would be multiplied by the Jacobian of the transformation from $\log\sigma$ to $\sigma^2$ and the mode would change slightly, to $\tilde{\sigma}^2 = \frac{n}{n+2}\hat{\sigma}^2$.

My question is how do we compute the Jacobian from $\log\sigma$ to $\sigma^2$? I can differentiate one function with respect to the other but I can't reconcile the stated change in mode (which makes me think I'm mistaken).

Thanks in advance

$\endgroup$

1 Answer 1

4
$\begingroup$

For the parameter transform$$\eta\longmapsto\sigma^2=\exp\{2\eta\}$$the Jacobian is $$\frac{\text d\sigma^2}{\text d\eta}=2\exp\{2\eta\}=2\sigma^2$$and the posterior changes from $p(\mu,\eta)$ into$$p^\prime(\mu,\sigma^2)=p(\mu,\log(\sigma^2)/2)\times \frac{\text d\eta}{\text d\sigma^2} = p(\mu,\log(\sigma^2)/2) \frac{1}{2\sigma^2}$$which modifies the location of the mode: $$\arg\max_{\mu,\sigma^2}p(\mu,\log(\sigma^2)/2)\ne \arg\max_{\mu,\sigma^2}p(\mu,\log(\sigma^2)/2)\frac{1}{2\sigma^2}$$ Namely, $$\hat\sigma^2 = \arg\max_{\sigma^2} \left\{-n\log \sigma-\frac{n \hat\sigma^2}{2\sigma^2}\right\}$$ versus $${\tilde\sigma}^2 = \arg\max_{\sigma^2}\left\{-n\log \sigma-\frac{n \hat\sigma^2}{2\sigma^2}\underbrace{-\log \sigma^2}_{+\log \frac{\text d\eta}{\text d\sigma^2}}\right\}=\frac{n}{n+2}\hat\sigma^2$$

$\endgroup$
3
  • $\begingroup$ Thank you Xi'an - this is very helpful. Minor follow-up: when I said I can't reconcile the stated change in mode I meant the specific stated change in the book i.e. to $\frac{n}{n+2}\hat{\sigma}^2$. $\endgroup$ Sep 27, 2020 at 14:57
  • $\begingroup$ Wonderful, thank you. Final check: is there a typo in the Jacobian, it should be $\frac{d \eta}{d \sigma^2}$? $\endgroup$ Sep 28, 2020 at 13:57
  • 1
    $\begingroup$ Both are. The Jacobian of one transform $\eta\mapsto\sigma^2$ is the inverse of the reverse transform $\sigma^2\mapsto\eta$. (Typo corrected.) $\endgroup$
    – Xi'an
    Sep 28, 2020 at 16:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.