The uniform distribution, with the support that has a finite measure, guarantees that the entropy is maximum(as stated in this answer), but in our daily life, normal distribution seems more uninformative(in this post the author said that a normal prior would always be preferable). We all know that noise is Gaussian-distributed. Is noise more noninformative than uniform?

  • 1
    $\begingroup$ For two reasons I find this question to be meaningless. (1) "Entropy is maximum" is not defined unless you specify what the support of the distribution is, and even then a maximum entropy distribution might not exist. (2) For "non-informative" to have any meaning, you need to give a specific parameterization of a distribution family in a Bayesian setting. $\endgroup$ – whuber Jan 14 at 14:13
  • $\begingroup$ @whuber I mean for any distribution family, either conjugate or non-conjugate distribution. $\endgroup$ – Lerner Zhang Jan 14 at 14:37
  • $\begingroup$ @whuber If I am not wrong, if the support is certain the uniform distribution would guarantee maximum entropy, right? $\endgroup$ – Lerner Zhang Jan 14 at 14:42
  • 1
    $\begingroup$ That is correct provided the support has finite measure. $\endgroup$ – whuber Jan 14 at 14:58
  • 1
    $\begingroup$ Re the edit: no Normal distribution has finite support, so it seems unlikely anyone would be seriously comparing uniform and Normal distributions. In particular, it is hard to determine how one might compare two such distributions in terms of how "noninformative" they might be. $\endgroup$ – whuber Jan 14 at 15:05