What kind of constrains give rise to bimodal distributions in the Maximum Entropy formalism? Are there any known results in this topic?


1 Answer 1


This might be related to the support of the constraints. With the following notation: let X be a discrete random variable on states, let P be a probability distribution with P(X=x_i)=p_i for i=1,...,n where n=length(states), and let H(X)=-sum(p_i*log(p_i) for i=1,...,n) be the entropy of X.

Define constraints by:

A_j: R^n -> R 

as a column vector of length n, j=1,...,m and demand that the conditions

A_j %*% X == bvec[j]  for j=1,...,m 

are satisfied.

Now if the constraints are supported on different subsets of states, a bimodal distribution may arise. For example:

n <- 40
p <- 0.5
mu <- n*p
A_1 <- c(1:n, rep(0,n))
A_2 <- c((1:n-mu)^2, rep(0,n))
A_3 <- c(rep(0,n), 1:n)
A_4 <- c(rep(0,n), (1:n-mu)^2)
bvec <- c(mu, n*p*(1-p), mu, n*p*(1-p))/2

The first two conditions result in a binomial distribution on 1:40 and the second two conditions result in a binomial distribution on 41:80. The maximum entropy distribution on states=1:80 has then two modi:

maximum entropy distribution of the above constraints

I believe this generalizes to continuous random variables.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.