2
$\begingroup$

I am working in a Bayesian framework: I have some observations $y$, for which I assume a statistical model. The model depends on parameters $\theta \in \Theta$ ($\Theta$ is the parameters space). I assume a probability distribution $q$ on $\Theta$. The parameters of this model can be estimated in a maximum a posteriori fashion : $$ \hat{\theta} = \mathop{\mathrm{argmax}} \limits_{\theta \in \Theta} p(\theta \mid y) $$ where $p(\theta \mid y)$ is the posterior distribution of $\theta$ given $y$. Now, say I want to perform a change of variable on $\Theta$. I consider a mapping $g \, : \, \Theta \, \rightarrow \, \Theta$ which transforms the "old" parameters $\theta$ in $\theta^{\mathrm{new}} = g(\theta)$. We assume that $g$ is a smooth diffeomorphism. My question is : how does this change of variable modifies the posterior distribution $p(\theta \mid y)$ ?

If $\mathrm{J}_{g}(\theta)$ denotes the jacobian matrix of $g$ at $\theta$, we know that $\theta^{\mathrm{new}}$ has a probability distribution $\widetilde{q}$ on $\Theta$ given by :

$$ \widetilde{q}(\theta^{\mathrm{new}}) \vert \mathrm{J}_{g}(\theta) \vert = q(\theta). $$

Using Bayes formula, I would write :

$$ p(\theta^{\mathrm{new}} \mid y) = \frac{ p\big( y \mid \theta^{\mathrm{new}} \big) \widetilde{q}(\theta^{\mathrm{new}}) }{ p(y) } = \frac{ p\big( y \mid g(\theta) \big) \vert \mathrm{J}_{g}(\theta) \vert^{-1} q(\theta) }{ p(y) }. $$

Is this correct or am I mistaken?

$\endgroup$
1
  • 2
    $\begingroup$ For any probability distribution the change of variables (that is differentiable, and one-to-one) leads to the new pdf that is the original pdf divided/multiplied by the Jacobian of the transformation. So the fact that your are dealing with the posterior distribution doesn't make any difference. $\endgroup$
    – sega_sai
    Nov 24, 2016 at 15:00

1 Answer 1

2
$\begingroup$

The change of variable in the posterior density is a standard change of variable, involving the Jacobian. The impact on the maximum a posteriori estimator is thus significant in that the MAP of the transform is not the transform of the MAP. (There are deeper reasons for disliking MAP estimators, of course.)

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.