6
$\begingroup$

Suppose I have a distribution which belongs to an exponential family, of the form

$$p(x) = \frac{\exp(-\sum_k \eta_k T_k(x))}{Z},$$

where $T_k(x)$ are a fixed set of sufficient statistics, $\eta_k$ are the corresponding natural parameters, and $Z$ is the partition function.

The functional form of the sufficient statistics $T_k$ are all known, so when I have data, I can calculate the values of all the sufficient statistics. However, I don't know the natural parameters.

Despite this, I am wondering: is there some known algorithm for producing samples from such a distribution, knowing only the values of the sufficient statistics?

[In particular, I am interested in an energy-based distribution $p(x) = \frac{\exp(-E(x))}{Z}$.]

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ I would clarify your point that "all sufficient statistics" are known. A random sample of $n$ points is, as a whole, a sufficient statistic but surely you don't mean all possible random samples. Do you mean minimal sufficient statistics? Even then, minimal sufficient statistics are functions of the data, so do you have actual numbers based on data or do you have the function such that when you gather data, you can calculate the value of the statistic? For example, Do you know the sufficient statistic is $\bar{X}$ or do you actually have the value $\bar{x}$? $\endgroup$
    – Matt Brems
    Jan 7, 2016 at 16:37
  • $\begingroup$ Edited to provide more information. Let me know if it is still unclear $\endgroup$
    – co9olguy
    Jan 7, 2016 at 17:03

1 Answer 1

Reset to default
3
$\begingroup$

Given that the vector $\mathbf{T}(X)=(T_1(X),\ldots,T_K(X))$ is sufficient, this means that the distribution of $X$ conditional on $\mathbf{T}(X)$ does not depend on the parameters $\eta_k$.

If $$p(x)\propto h(x)\exp\left\{-\sum_k \eta_k T_k(x)\right\}$$ the distribution of $X$ given $\mathbf{T}(X)=t$ will have a density proportional to $h(x)$ on the manifold $\mathbf{T}(x)=t$.

I am not aware of a generic algorithm to handle this simulation. In the case of the energy function, the distribution of $X$ is then uniform over the set of $x$'s such that $E(x)=e$, the observed value of the sufficient statistic.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.