Sampling parameters from exponential family

Question

So suppose PDF $f_{X|\theta}(x_1,...,x_n;\theta_1,...,\theta_m)$ is from the exponential family. Is there any theory or general guidelines for sampling parameters from this PDF?

This question is not about sampling methods like Gibbs, Metropolis-Hastings, Importance, Slice, etc. Rather I wonder whether I could exploit the fact that $X$ is from the exponential family such as to write an efficient sampler for $\theta$?

$$ f_{\theta|X}(\theta|x) = \frac{f_{X|\theta}(x|\theta)f(\theta)}{f(x)} $$

Lets just suppose that $f(\theta)$ is uniform. In that case this is equivalent as sampling $\theta$ from the likelihood $L_{\theta|X}(\theta;x)$ in the frequentist framework.

So this is a very general and theoretical question. Ideas and pointers are appreciated.

Answer 1

The question is imprecise as stated as it depends on the parameterisation of the exponential family. If considering a generic representation of the density of this exponential family $$f_{X|\theta}(x|\theta)=h(x)\exp\{R(\theta)\cdot S(x)-\psi(\theta)\}$$ wrt a given dominating measure, there is no reason for $\theta$ to be easily simulated from the target density $$\pi(\theta|x)\propto\exp\{R(\theta)\cdot S(x)-\psi(\theta)\}$$ (assuming the dominating measure is such that this density can be normalised into a probability density). In the specific case when the natural parameterisation of the exponential family is used, $$f_{X|\theta}(x|\theta)=h(x)\exp\{\theta\cdot S(x)-\psi(\theta)\},$$ it may get easier to simulate $\theta$ but this is not necessarily the case. A simple counter-example is when $\theta=(\alpha,\beta)\in(0,1)^2$ is the standard parameter of a Beta distribution $$f_{X|\theta}(x|\theta)=\dfrac{\Gamma(\alpha+\beta)}{\Gamma(\alpha)\Gamma(\beta)}x^{\alpha-1}(1-x)^{\beta-1}$$ or $$f_{X|\theta}(x|\theta)=\dfrac{\Gamma(\alpha+\beta)^n}{\Gamma(\alpha)^n\Gamma(\beta)^n}\underbrace{\prod_{i=1}^n x_i^{\alpha-1}}_{A^{\alpha-1}}\underbrace{\prod_{i=1}^n(1-x_i)^{\beta-1}}_{B^{\beta-1}}$$ for a sample. There is no shortcut to simulate $$\pi(\theta)\propto \dfrac{\Gamma(\alpha+\beta)^n}{\Gamma(\alpha)^n\Gamma(\beta)^n}{A^{\alpha}}{B^{\beta}}$$ (against the Lebesgue measure on $(\mathbb{R}_+^*)^2$).