From bayesian theory, we have that if $f(x|\eta) \propto \exp(\eta \cdot T(x)- A(\eta))$ - a natural exponential family, then the prior conjugate of $\eta$ is $\pi^*(\eta | \mu, \lambda) \propto \exp(\mu \cdot x- \lambda A(\eta))$ and the posterior is $\pi^*(\eta|x) = \pi(\eta | \mu + T(x), \lambda + 1)$ [1].

I am confused by my bayesian professor (in a undergrad class) with respect of findind these parameters, $\mu$ and $\lambda$.

My question is: when we start with "standard" parameterization $f(x|\theta) \propto \exp(B(\theta) \cdot T(x) - A(\theta))$ and then reparameterize to natural parameter $\eta$ as above with $\eta = B(\theta)$, does $\pi(\theta | \mu, \lambda) = |B'(\theta)| \pi^*(B(\theta) | \mu, \lambda)$? Or does $\pi(\theta | \mu, \lambda) \propto \pi^*(B(\theta) | \mu, \lambda)$?

In other words, we can only substitute $B(\theta)$ on $\eta$? Or do we need to consider the jacobian, since $B(\theta)$ is a transformation of the random variable $\theta$?

If only substitution is enough, then we conclude that jacobian does not change the family of $\pi$. And in that case, the "correct" hyperparameters $\mu$ and $\lambda$ are not of importance to bayesian inference. But in that case, the table from the reference book [1] does not hold.


For example, if $x \sim Poisson(\theta)$ then $f(x | \theta) \propto \exp\{B(\theta)x - \theta\}$, where $B(\theta) = \ln(\theta)$. Hence $\eta = \ln(\theta)$ and $f(x | \eta) \propto \exp\{\eta x - e^{\eta}\}$. Therefore $\pi^*(\eta | \mu, \lambda) \propto \exp\{\mu \eta - \lambda e^{\eta}\}$.

Now we have $|B'(\theta)| = \theta^{-1}$ and $\pi(\theta | \mu, \lambda) \propto \theta^{\mu-1} e^{-\lambda \theta}$, which is Gamma$(\mu, \lambda)$. If we do not consider the jacobian, $\pi(\theta | \mu, \lambda) \propto \theta^{\mu} e^{-\lambda \theta}$, which is Gamma$(\mu+1, \lambda)$.

So there is a correct parameterization? Notice that both of them follow the posterior of this table (under a reparameterization to $\alpha$ and $\beta$).

[1] Robert, Christian. The Bayesian choice: from decision-theoretic foundations to computational implementation. Springer Science & Business Media, 2007.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.