5
$\begingroup$

I can readily define asymptotic optimality but I find the notion of Bayes optimality harder to pin down. What is a correct and general mathematical definition of Bayes Optimality?

Note: The notion of asymptotic optimality I have in mind is that in the limit of an infinite amount of data the statistical system converges to the optimal decision. Deep Q-learning is an example of such a method. By Bayes-Optimal my intuition is that it must mean that our system always makes the optimal decision given the available information.

$\endgroup$
  • $\begingroup$ What do you mean by asymptotic? Do you mean something like ''if I get more and more data I'll come closer and closer to the real result'' to be asymptotic? Because that is in fact what people seem to understand as Bayes optimal (or Bayes consistent, i.e. emprical risk converges to true best risk, see arxiv.org/pdf/0810.4752.pdf)... If you want some formulation that is 'not asymptotic' or 'general' as you call it, what do you expect it to be? $\endgroup$ – Fabian Werner Aug 29 '17 at 10:43
  • $\begingroup$ @FabianWerner By Bayes-Optimal my intuition is that it must mean that our system always makes the optimal decision relative to the available information. Is this a reasonable definition? $\endgroup$ – Aidan Rocke Aug 29 '17 at 11:12
  • 1
    $\begingroup$ I'd say yes... in the sense that the overall Risk (or Error or whatever you want to call it) function measuring how 'wrong' the predictions were is minimized.... like in the pdf: X is the input, Y the true output and f our model then L(X, Y, f) = {1 if f(X) != Y and 0 otherwise}. Then R(f) = EL(X,Y,f(X)) measures how well f performs (i.e. R(f) big --> f performs poorly, R(f) = 0 --> f performs perfectly). Then any function g is called Bayes Optimal iff. R(g) <= R(f) for all functions f in the function space under consideration... What about that? $\endgroup$ – Fabian Werner Aug 29 '17 at 11:26
  • 1
    $\begingroup$ @FabianWerner That seems like a good definition. You should post it as an answer. $\endgroup$ – Aidan Rocke Aug 29 '17 at 11:50
3
$\begingroup$

Bayes optimal estimator is any estimator that minimizes the Bayes risk, i.e. the expected posterior loss.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.