0
$\begingroup$

From bayesian theory, we have that if $f(x|\eta) \propto \exp(\eta \cdot T(x)- A(\eta))$ - a natural exponential family, then the prior conjugate of $\eta$ is $\pi^*(\eta | \mu, \lambda) \propto \exp(\mu \cdot x- \lambda A(\eta))$ and the posterior is $\pi^*(\eta|x) = \pi(\eta | \mu + T(x), \lambda + 1)$ [1].

I am confused by my bayesian professor (in a undergrad class) with respect of findind these parameters, $\mu$ and $\lambda$.

My question is: when we start with "standard" parameterization $f(x|\theta) \propto \exp(B(\theta) \cdot T(x) - A(\theta))$ and then reparameterize to natural parameter $\eta$ as above with $\eta = B(\theta)$, does $\pi(\theta | \mu, \lambda) = |B'(\theta)| \pi^*(B(\theta) | \mu, \lambda)$? Or does $\pi(\theta | \mu, \lambda) \propto \pi^*(B(\theta) | \mu, \lambda)$?


In other words, we can only substitute $B(\theta)$ on $\eta$? Or do we need to consider the jacobian, since $B(\theta)$ is a transformation of the random variable $\theta$?

If only substitution is enough, then we conclude that jacobian does not change the family of $\pi$. And in that case, the "correct" hyperparameters $\mu$ and $\lambda$ are not of importance to bayesian inference. But in that case, the table from the reference book [1] does not hold.

.

For example, if $x \sim Poisson(\theta)$ then $f(x | \theta) \propto \exp\{B(\theta)x - \theta\}$, where $B(\theta) = \ln(\theta)$. Hence $\eta = \ln(\theta)$ and $f(x | \eta) \propto \exp\{\eta x - e^{\eta}\}$. Therefore $\pi^*(\eta | \mu, \lambda) \propto \exp\{\mu \eta - \lambda e^{\eta}\}$.

Now we have $|B'(\theta)| = \theta^{-1}$ and $\pi(\theta | \mu, \lambda) \propto \theta^{\mu-1} e^{-\lambda \theta}$, which is Gamma$(\mu, \lambda)$. If we do not consider the jacobian, $\pi(\theta | \mu, \lambda) \propto \theta^{\mu} e^{-\lambda \theta}$, which is Gamma$(\mu+1, \lambda)$.

So there is a correct parameterization? Notice that both of them follow the posterior of this table (under a reparameterization to $\alpha$ and $\beta$).

[1] Robert, Christian. The Bayesian choice: from decision-theoretic foundations to computational implementation. Springer Science & Business Media, 2007.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.