I'm trying to devise a prior for a model parameter $x$ about which I know the following things:

  1. It is strictly positive.
  2. There is a maximum possible value $x_m$.
  3. Larger values are less likely than smaller ones.

In terms of a probability distribution $P(x)$, I interpret these as

  1. $P(x)=0$ if $x<0$.
  2. $P(x)=0$ if $x>x_m$
  3. $P'(x)<0$ for $0<x<x_m$.

One option is to just use a straight line that declines to zero at $x_m$, which would give


I guess this is basically a beta distribution with $\alpha=1$ and $\beta=2$. Without futher guidance, the linearity seems like a pretty strong assumption and I can't justify it. I realise that having a mode$=0$ would force a beta distribution to have $\alpha=1$ and $P'(x)<0$ requires $\beta>1$, but I don't see what value of $\beta$ is best, nor any reason to want a beta distribution anyway.

What I'd like to do is use the Maximum Entropy principle, but I can't really work out how to make it work. I've been following the textbook by Gregory but I don't really know how to incorporate these requirements. The examples of continuous distributions are always derived as limits of discrete ones, so I tried starting with that. I describe the monoticity by the $n$ constraints

$$\frac{\partial x_i}{\partial p_i}+k_i=0$$

where $p_i$ is the probability of the value $x_i$, and $k_i$ is some positive number representing the slope of the distribution at $x_i$. Then, the entropy is maximized by finding

$$d\left(\sum p_i\ln p_i-\lambda\left(\sum p_i-1\right)-\sum\lambda_i\left(\partial x_i/\partial p_i+k_i\right)\right)=0$$

from which I eventually get to

$$-\ln p_i=1+\lambda+\lambda_i\left(\partial^2 x_i/\partial p_i^2+\partial k_i/\partial p_i\right)$$

I really don't know how to proceed from here. More likely is that I'm on completely the wrong track, but I thought it might help if I could show you what I tried.

So, to cut it short, my question is, what is the MaxEnt distribution that satisfies the requirements above?

  • $\begingroup$ Restriction (3) cannot possibly be binding for your beta distributions--it's a strict inequality. The extremum must be reached when the derivative is identically at its limiting value of constantly zero: that's a uniform distribution (or scaled Beta$(1,1)$, if you like). I leave it to you to decide whether this gives a maximum or minimum solution :-). $\endgroup$ – whuber Feb 21 '13 at 8:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.