# Sampling discrete distributions given their entropy

Can we (uniformly) generate a discrete random variable over $$n$$ possible outcomes given its entropy? I am interested in non-parametric methods, that do not require the random variable to follow a specific distribution.

This problem is equivalent to finding $$\{x_i\}_{1 \leq i \leq n}$$ with:

$$\sum_{i=1}^n x_i = 1$$ and: $$-\sum_{i=1}^n x_i \, \text{log}_2(x_i) = \text{given entropy}$$

Finding one discrete distribution with a given entropy is not difficult (using a greedy algorithm) but uniformly sampling one from all potential distributions seems much harder.

• Can you tell us something about the context in which this arises ... – kjetil b halvorsen Dec 17 '16 at 18:43
• Yes, the final goal is to analyze multivariate data sets of discrete data, and find an approximate model to generate them given a few parameters. Here, the problem consists of generating one discrete distribution (one column) given a few parameters (entropy, number of values, etc.). – cynddl Jan 16 '17 at 13:52