Questions: Is the AUC (area under the ROC curve) a type of "empirical Bayes estimator"?

If we take the parameter space to be $\Theta = [0,1]$ and the prior $\Lambda$ to be Lebesgue measure, then the Bayes estimator just gives the estimator whose risk function's curve has the smallest area.

So if there is anyway to identify ROC curves with the curves of some "empirical risk function", then it seems like there might be some sort of connection.

Definitions: Here are the definitions from my course:

  • A statistical model is a family of candidate probability distributions $$\mathcal{P} = \{ P_{\theta}: \theta \in \Theta \} $$ for some observed data $X \sim P_{\theta}$.
  • The Bayes estimator $\delta$ with respect to the prior $\Lambda$ on the parameter space $\Theta$ gives the minimized average-case risk over some measure $\Lambda$ on the parameter space $\Theta$: $$\min_{\delta}\int_{\Theta} R(\theta, \delta)\ \mathrm{d}\Lambda(\theta) \,. $$
| cite | improve this question | | | | |
  • $\begingroup$ I think this stats.stackexchange.com/questions/189411/… thread is closely related to your question. $\endgroup$ – Tim Aug 29 '17 at 22:23
  • $\begingroup$ @Tim You are right that that question does come up as a suggestion. My professor said that the Bayes estimator isn't necessarily "Bayesian" but also used in frequentist/classical statistics (don't know if that's true, that's just what they said). For example, the word "risk" doesn't show up in their question at all, so maybe it isn't about risk functions like this question is. $\endgroup$ – Chill2Macht Aug 29 '17 at 23:08
  • 1
    $\begingroup$ Usually, empirical Bayes refers to a type of Bayesian inference where the prior is coming from data, i.e. 'empirical prior'. $\endgroup$ – Julius Aug 29 '17 at 23:11
  • $\begingroup$ To whoever downvoted, would you mind explaining the downvote? It was a legitimate question I had when I was first learning about these things. Downvoting without providing suggestions for improvement is not constructive, making anyone who does so a net detractor from the website. $\endgroup$ – Chill2Macht Dec 8 '17 at 18:09

Choosing the estimator with the least area under the curve is just choosing the Bayes estimator for the flat prior (i.e. the improper prior which is Lebesgue measure).

It is well-known that using the flat prior is mathematically equivalent to maximum likelihood estimation. One can interpret it philosophically in many ways, but it is the same procedure.

Since the empirical distribution is the (non-parametric) maximum likelihood estimate for the true distribution, then perhaps using maximum likelihood estimation for parametric problems could also in some sense be considered, or at least be described as, 'empirical'.

Moreover, what is called an empirical Bayes procedure, as mentioned in the comments, starts with the maximum likelihood estimate of the parameter as the input. Since the least area under the curve procedure mentioned in the question is just the maximum likelihood estimate, it follows that it actually is the same thing as empirical Bayes, even though I had not heard of that term at the time when I had asked the question originally (several months ago).

| cite | improve this answer | | | | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.