1
$\begingroup$

We know that the normal distribution has the maximum entropy among all continuous distribution on $\mathbb{R}$ for a given variance.

I wonder what's the opposite, i.e. what distribution has the maximum (or minimum) variance for a given entropy?

$\endgroup$
5
$\begingroup$

The normal distribution has the maximum entropy among all continuous distributions with fixed mean and variance on real support.

The variance for a given entropy can be made arbitrarily large using a mixture of two Gaussians that are spread farther and farther apart. The variance increases, while the entropy doesn't change much.

The minimum variance for a given entropy sounds the same to me as asking for the maximum entropy for a given variance, and so I think you're right that it is normal.

$\endgroup$
  • $\begingroup$ Thanks, I missed the "for given variance" part in the question, which has been edited. My haunch for the minimum variance is the normal distribution, but i am not sure. $\endgroup$ – chaohuang Feb 13 '13 at 21:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.