3
$\begingroup$

I see a lot of implementation of maximum entropy. Can someone explain why it is possible to estimate the parameter based on maximum entropy?

$\endgroup$
3
  • 3
    $\begingroup$ Answers to this question aren't likely to be helpful or constructive. It is possible because entropy is bounded above and almost always is a continuous function of the parameter. Thus, by varying the parameter withing a bounded region you can always find a maximum. A more penetrating avenue of investigation would be to ask what desirable properties a maximum-entropy estimator might have. $\endgroup$
    – whuber
    May 12 '17 at 21:42
  • 2
    $\begingroup$ I want to get some sense on maximum entropy. As you mentioned, the entropy is bounded above. And it's common to use the maximum entropy as good criteria to estimate the hidden parameters. Can you explain the reason why it is reasonable to pick up estimated parameter by virtue of reaching the peak on entropy plot? In other words, how to connect the maximum-entropy to other easily understood criteria together, like maximum likelihood estimation? Thx $\endgroup$
    – Sirius Hou
    May 13 '17 at 2:58
  • $\begingroup$ Some ideas here: stats.stackexchange.com/questions/66186/… $\endgroup$ Jun 5 '17 at 22:58