Suppose I have a distribution which belongs to an exponential family, of the form

$$p(x) = \frac{\exp(-\sum_k \eta_k T_k(x))}{Z},$$

where $T_k(x)$ are a fixed set of sufficient statistics, $\eta_k$ are the corresponding natural parameters, and $Z$ is the partition function.

The functional form of the sufficient statistics $T_k$ are all known, so when I have data, I can calculate the values of all the sufficient statistics. However, I don't know the natural parameters.

Despite this, I am wondering: is there some known algorithm for producing samples from such a distribution, knowing only the values of the sufficient statistics?

[In particular, I am interested in an energy-based distribution $p(x) = \frac{\exp(-E(x))}{Z}$.]

  • 1
    $\begingroup$ I would clarify your point that "all sufficient statistics" are known. A random sample of $n$ points is, as a whole, a sufficient statistic but surely you don't mean all possible random samples. Do you mean minimal sufficient statistics? Even then, minimal sufficient statistics are functions of the data, so do you have actual numbers based on data or do you have the function such that when you gather data, you can calculate the value of the statistic? For example, Do you know the sufficient statistic is $\bar{X}$ or do you actually have the value $\bar{x}$? $\endgroup$
    – Matt Brems
    Commented Jan 7, 2016 at 16:37
  • $\begingroup$ Edited to provide more information. Let me know if it is still unclear $\endgroup$
    – co9olguy
    Commented Jan 7, 2016 at 17:03

2 Answers 2


Given that the vector $\mathbf{T}(X)=(T_1(X),\ldots,T_K(X))$ is sufficient, this means that the distribution of $X$ conditional on $\mathbf{T}(X)$ does not depend on the parameters $\eta_k$.

If $$p(x)\propto h(x)\exp\left\{-\sum_k \eta_k T_k(x)\right\}$$ the distribution of $X$ given $\mathbf{T}(X)=t$ will have a density proportional to $h(x)$ on the manifold $\mathbf{T}(x)=t$.

I am not aware of a generic algorithm to handle this simulation. In the case of the energy function, the distribution of $X$ is then uniform over the set of $x$'s such that $E(x)=e$, the observed value of the sufficient statistic.


This paper looks exactly at this question?

Use of the Gibbs Sampler to Obtain Conditional Tests, with Applications by Lockhart, O'Reilly and Stephens


have a look!

  • $\begingroup$ While this link may answer the question, it is better to include the essential parts of the answer here and provide the link for reference. Link-only answers can become invalid if the linked page changes. - From Review $\endgroup$ Commented Feb 8 at 0:36
  • $\begingroup$ This could use some supporting information and how it solves the problem presented by OP. $\endgroup$ Commented Feb 8 at 0:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.