Revisions to Bayesian vs Maximum entropy

edited tags

Link

edited Jun 5, 2017 at 23:00

kjetil b halvorsen ♦

82.8k
32
201
663

deleted 176 characters in body

Source Link

edited Mar 9, 2011 at 14:09

Ashok

1.2k
9
24

Following is a question I asked in AoPS forum and I was informed that this would be a better place to ask this question. Here's the question:

Suppose that the quantity which we want to infer is a probability distribution. All we know is that the distribution comes from a set $E$ determined, say, by some of its moments and we have a prior $Q$.

The maximum entropy principle(MEP) says (supported by Sanov's theorem) that the $P^{\star}\in E$ which has least relative entropy from $Q$ (i.e., $P^{\star}=\displaystyle \text{argmin}_{P\in E}D(P\|Q)$) is the best one to select. Whereas the Bayesian rule of selection has a process of selecting the posterior given the prior which is supported by Bayes' theorem.

My question is whether there is any connection between these two inference methods (i. e., whether the two methods apply to the same problem and have something in common)? Or whether in Bayesian inference the setting is completely different from the above mentioned setting? Or am I not making sense?!

Following is a question I asked in AoPS forum and I was informed that this would be a better place to ask this question. Here's the question:

Suppose that the quantity which we want to infer is a probability distribution. All we know is that the distribution comes from a set $E$ determined, say, by some of its moments and we have a prior $Q$.

The maximum entropy principle(MEP) says (supported by Sanov's theorem) that the $P^{\star}\in E$ which has least relative entropy from $Q$ (i.e., $P^{\star}=\displaystyle \text{argmin}_{P\in E}D(P\|Q)$) is the best one to select. Whereas the Bayesian rule of selection has a process of selecting the posterior given the prior which is supported by Bayes' theorem.

My question is whether there is any connection between these two inference methods (i. e., whether the two methods apply to the same problem and have something in common)? Or whether in Bayesian inference the setting is completely different from the above mentioned setting? Or am I not making sense?!

Suppose that the quantity which we want to infer is a probability distribution. All we know is that the distribution comes from a set $E$ determined, say, by some of its moments and we have a prior $Q$.

The maximum entropy principle(MEP) says that the $P^{\star}\in E$ which has least relative entropy from $Q$ (i.e., $P^{\star}=\displaystyle \text{argmin}_{P\in E}D(P\|Q)$) is the best one to select. Whereas the Bayesian rule of selection has a process of selecting the posterior given the prior which is supported by Bayes' theorem.

My question is whether there is any connection between these two inference methods (i. e., whether the two methods apply to the same problem and have something in common)? Or whether in Bayesian inference the setting is completely different from the above mentioned setting? Or am I not making sense?!