Maximum entropy priors for hypotheses

Question

Suppose I have a random variable X which can take integer values 0 through 99. It can be, for example, numbers written on balls in a large urn. I have three hypotheses about the distribution of these numbers on the balls: $H_0$ says each number is just as likely as every other number; $H_1$ says that there's a 90% chance that the number on any given ball is between 50 and 59 including; and $H_2$ says that there's a 90% chance that the number on any given ball is 55.

I want to know, based on the Principle of Maximum Entropy, what the priors for each of these hypotheses should be, and how to calculate that. It feels intuitive to me that $P(H_1):P(H_2)$ should equal $10:1$ or something like that but I don't know how to show that.

Answer 1

Technically, if you want to use the "maximum" entropy principle, then in this case you would simply choose the hypothesis with the maximum entropy, which would be the $H_0$, the uniform distribution.

An alternate approach, motivated by statistical thermodynamics, would be to weight the priors according to their exponentiated entropy, so $\mathcal P[H_i]\propto e^{\mathcal H(H_i)}$, where $$\mathcal H(H_i)=-\sum_{j=1}^{100} p_{H_i}(j)\log p_{H_i}(j).$$

It is easy to compute: $$\begin{align} \mathcal H(H_0) &= \log 100 = 4.60517\\ \mathcal H(H_0) &= \log 900 -\frac{810\log 81 + 90\log 1}{900} = 2.847391\\ \mathcal H(H_0) &= \log 990 - \frac{891\log 891 + 99\log 1}{990} = 0.784595\\ \end{align}$$

Exponentiating and normalizing yields $\mathcal P(H_0) = 0.83728$, $\mathcal P(H_1) = 0.14437$, $\mathcal P(H_2) = 0.01835$.