I am interested in the Bayes risk of some distribution $\pi$ $$ r(\pi) = \mathbb{E}_{\pi(x)}[ \mathbb{E}_{\Pr(y|d,x)}[L(x,\hat x(y|d))]], $$ where $L$ is some loss function and $\hat x$ is the posterior mean (here: Bayes estimator). Here $d$ is some experimental design parameter.

If $L(x,\hat x) = |x-\hat x|^2$ (squared error), then the Posterior Cramer-Rao bound gives a lower bound on the Bayes risk. The Bayes risk is also identical to the expected posterior variance. So to optimally design experiments, I can use negative expected variance as a utility function to maximize.

Now I would like to reverse this logic to find the effective loss function and a lower bound for information gain as an alternative utility function. That is, suppose the utility function I use is information gain. Then,

  1. What is the loss function which this is optimizing?
  2. What is a lower bound on the Bayes risk for this loss function?
  • $\begingroup$ Generalization bounds are in this spirit: videolectures.net/mlss05us_langford_gb $\endgroup$ – Tristan Apr 24 '12 at 7:50
  • $\begingroup$ @Tristan - Thanks. Interesting and certainly seems relevant. Although, I'm not sure yet whether it answers my specific question--maybe I just need a better question, though. $\endgroup$ – Chris Ferrie May 4 '12 at 11:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.