3
$\begingroup$

So suppose PDF $f_{X|\theta}(x_1,...,x_n;\theta_1,...,\theta_m)$ is from the exponential family. Is there any theory or general guidelines for sampling parameters from this PDF?

This question is not about sampling methods like Gibbs, Metropolis-Hastings, Importance, Slice, etc. Rather I wonder whether I could exploit the fact that $X$ is from the exponential family such as to write an efficient sampler for $\theta$?

$$ f_{\theta|X}(\theta|x) = \frac{f_{X|\theta}(x|\theta)f(\theta)}{f(x)} $$

Lets just suppose that $f(\theta)$ is uniform. In that case this is equivalent as sampling $\theta$ from the likelihood $L_{\theta|X}(\theta;x)$ in the frequentist framework.

So this is a very general and theoretical question. Ideas and pointers are appreciated.

$\endgroup$
  • 1
    $\begingroup$ What do you mean by "sampling parameters from this pdf"? $\endgroup$ – Glen_b Jun 18 '18 at 8:59
  • $\begingroup$ @Glen_b, good point. I updated the question. $\endgroup$ – Davor Josipovic Jun 18 '18 at 9:03
2
$\begingroup$

The question is imprecise as stated as it depends on the parameterisation of the exponential family. If considering a generic representation of the density of this exponential family $$f_{X|\theta}(x|\theta)=h(x)\exp\{R(\theta)\cdot S(x)-\psi(\theta)\}$$ wrt a given dominating measure, there is no reason for $\theta$ to be easily simulated from the target density $$\pi(\theta|x)\propto\exp\{R(\theta)\cdot S(x)-\psi(\theta)\}$$ (assuming the dominating measure is such that this density can be normalised into a probability density). In the specific case when the natural parameterisation of the exponential family is used, $$f_{X|\theta}(x|\theta)=h(x)\exp\{\theta\cdot S(x)-\psi(\theta)\},$$ it may get easier to simulate $\theta$ but this is not necessarily the case. A simple counter-example is when $\theta=(\alpha,\beta)\in(0,1)^2$ is the standard parameter of a Beta distribution $$f_{X|\theta}(x|\theta)=\dfrac{\Gamma(\alpha+\beta)}{\Gamma(\alpha)\Gamma(\beta)}x^{\alpha-1}(1-x)^{\beta-1}$$ or $$f_{X|\theta}(x|\theta)=\dfrac{\Gamma(\alpha+\beta)^n}{\Gamma(\alpha)^n\Gamma(\beta)^n}\underbrace{\prod_{i=1}^n x_i^{\alpha-1}}_{A^{\alpha-1}}\underbrace{\prod_{i=1}^n(1-x_i)^{\beta-1}}_{B^{\beta-1}}$$ for a sample. There is no shortcut to simulate $$\pi(\theta)\propto \dfrac{\Gamma(\alpha+\beta)^n}{\Gamma(\alpha)^n\Gamma(\beta)^n}{A^{\alpha}}{B^{\beta}}$$ (against the Lebesgue measure on $(\mathbb{R}_+^*)^2$).

$\endgroup$
  • 1
    $\begingroup$ The omission of $R(\theta)$ specification was intentional - so to not reduce it to my specific case. But your answer is very clear, educative and much appreciated @Xi'an! $\endgroup$ – Davor Josipovic Jun 18 '18 at 20:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.