Can we (uniformly) generate a discrete random variable over $n$ possible outcomes given its entropy? I am interested in non-parametric methods, that do not require the random variable to follow a specific distribution.

This problem is equivalent to finding $\{x_i\}_{1 \leq i \leq n}$ with:

$$\sum_{i=1}^n x_i = 1$$ and: $$-\sum_{i=1}^n x_i \, \text{log}_2(x_i) = \text{given entropy}$$

Finding one discrete distribution with a given entropy is not difficult (using a greedy algorithm) but uniformly sampling one from all potential distributions seems much harder.

  • 2
    $\begingroup$ Can you tell us something about the context in which this arises ... $\endgroup$ – kjetil b halvorsen Dec 17 '16 at 18:43
  • $\begingroup$ Yes, the final goal is to analyze multivariate data sets of discrete data, and find an approximate model to generate them given a few parameters. Here, the problem consists of generating one discrete distribution (one column) given a few parameters (entropy, number of values, etc.). $\endgroup$ – cynddl Jan 16 '17 at 13:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.