# Maximum entropy and non-informative distribution

Is Maximum Entropy rule equivalent to non-informativeness?

In other words, when maximizing the entropy of a distribution, given some known stuff, is it equivalent to finding to most non-informative distributions? If so, how do you justify this mathematically?

• It is as non-informative as it can mathematically be, given constraints on ignorance imposed by information. I've found John Harte's explanation in "Maximum Entropy and Ecology" to be quite accessible. Aug 16 '13 at 5:27
• This will sound pedantic, but maximum entropy distributions are non-informative whenever you interpret entropy as "a lack of information". In many situations this interpretation makes sense, but interpretations lie outside the scope of what can be justified with mathematics. Aug 16 '13 at 6:55
• +1. For example, many people talk informally about (say) extracting the information from the data, and this often is a matter of what is scientifically or practically interesting or important and scientists and practitioners often have a good sense of that. But it is crucial here that "information" is not defined, and there are many contexts in which defining it precisely would rid us of a useful word. We could always find another word, but "informative" will not always mean "in an information theory sense" in statistical discussions. Aug 16 '13 at 7:53
• Yeah, all of this make sense, but I am looking for something more formal. May be asking for too much .... Aug 16 '13 at 13:29