I'm reading Bishop's Pattern Recognition and Machine Learning. In chapter 1.6: Information Theory (page 53) when trying to derive the maximum differential entropy pdf from the definition of continuous entropy, the author states:
"Let us now consider the maximum entropy configuration for a continuous variable. In order for this maximum to be well defined, it will be necessary to constrain the first and second moments of p(x) as well as preserving the normalization constraint" (where p(x) is the pdf we wish to learn the form of)
My two questions:
- What does it mean for a maximum to be well-defined? Intuition + math please!
- Why does constraining the first and second moments of p(x) ensure this maximum is well defined - i.e., why do we not need to constrained all moments.