5
$\begingroup$

I can readily define asymptotic optimality but I find the notion of Bayes optimality harder to pin down. What is a correct and general mathematical definition of Bayes Optimality?

Note: The notion of asymptotic optimality I have in mind is that in the limit of an infinite amount of data the statistical system converges to the optimal decision. Deep Q-learning is an example of such a method. By Bayes-Optimal my intuition is that it must mean that our system always makes the optimal decision given the available information.

Improve this question
$\endgroup$
4
  • $\begingroup$ What do you mean by asymptotic? Do you mean something like ''if I get more and more data I'll come closer and closer to the real result'' to be asymptotic? Because that is in fact what people seem to understand as Bayes optimal (or Bayes consistent, i.e. emprical risk converges to true best risk, see arxiv.org/pdf/0810.4752.pdf)... If you want some formulation that is 'not asymptotic' or 'general' as you call it, what do you expect it to be? $\endgroup$
    – Fabian Werner
    Commented Aug 29, 2017 at 10:43
  • $\begingroup$ @FabianWerner By Bayes-Optimal my intuition is that it must mean that our system always makes the optimal decision relative to the available information. Is this a reasonable definition? $\endgroup$
    – Aidan Rocke
    Commented Aug 29, 2017 at 11:12
  • 1
    $\begingroup$ I'd say yes... in the sense that the overall Risk (or Error or whatever you want to call it) function measuring how 'wrong' the predictions were is minimized.... like in the pdf: X is the input, Y the true output and f our model then L(X, Y, f) = {1 if f(X) != Y and 0 otherwise}. Then R(f) = EL(X,Y,f(X)) measures how well f performs (i.e. R(f) big --> f performs poorly, R(f) = 0 --> f performs perfectly). Then any function g is called Bayes Optimal iff. R(g) <= R(f) for all functions f in the function space under consideration... What about that? $\endgroup$
    – Fabian Werner
    Commented Aug 29, 2017 at 11:26
  • 1
    $\begingroup$ @FabianWerner That seems like a good definition. You should post it as an answer. $\endgroup$
    – Aidan Rocke
    Commented Aug 29, 2017 at 11:50

1 Answer 1

Reset to default
3
$\begingroup$

Bayes optimal estimator is any estimator that minimizes the Bayes risk, i.e. the expected posterior loss.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.