Method 1

I am given a classifier for some disease that takes as input patient characteristics and has some sensitivity and specificity.

Hence the classifier is a function c(patient characteristics) = 1 or 0

I can then use Bayes rule to convert:

P(disease | c(patient characteristics) = 1) = P(c(patient characteristics) =1 |dz) P(dz) / P(c(patient characteristics) = 1)

Using classifier sensitivity and specificity to write P(c(patient characteristics) = 1 |dz)/P(c(patient characteristics) = 1).

So, even if all I have is a classifier (which made a decision), I can get some probability estimate.

Method 2

The better approach is to develop an estimator directly for

P(disease | patient characteristics)

Eg using logistic regression or just never binarizing classifier output in the first place.

Classifiers are heavily criticized in medicine, and I agree that it's a poor choice to make a decision without patient and physician utilities, but why can't we just use the first method to convert the classifiers to probabilities?


Don't overcomplicate your use of probabilistic reasoning by bringing in extraneous considerations. Regardless of the nature of the particular event (whether it involves patient characteristics or the outcome of a medical test) you can apply Bayes' rule to obtain the posterior probability that the patient has the disease. For any arbitrary event $\mathcal{A}$ you have:

$$\underbrace{\mathbb{P}(\text{Disease} | \mathcal{A})}_{\text{Posterior}} = \frac{\mathbb{P}(\mathcal{A} | \text{Disease} ) }{\mathbb{P}(\mathcal{A})} \cdot \underbrace{\mathbb{P}(\text{Disease})}_{\text{Prior}}.$$

Every event $\mathcal{A}$ that is statistically related to the event $\text{Disease}$ gives information that updates the probability of the latter according to Bayes' rule. Whether or not a medical test or a set of patient characteristics constitute better information depends on the data.

Update: Your additional edits to your question make it clear that you are also asking about the difference between conditioning on a vector of patient characteristics $\boldsymbol{x}$, versus a binary classifier $c(\boldsymbol{x})$ that maps the patients into two groups. The latter entails a loss of information, so you are correct that you are better off modelling based on the unclassified characteristics directly.

  • $\begingroup$ Thank you I edited to be more explicit, but I suspect your answer will be the same. $\endgroup$ – user0 Apr 27 '18 at 14:23
  • $\begingroup$ I have added an update to my answer to add more information responding to your edit. Hope this is helpful. $\endgroup$ – Ben - Reinstate Monica Apr 28 '18 at 3:40
  • $\begingroup$ Thank you, yes--your original answer actually pretty much answered in saying that method 1 is feasible, if I understand correctly. My question was really whether method 1 was reasonable (even if not optimal) since many times in medicine we are given a classification, or some result of a medical test, which I suppose is also a classification, but no posterior. So I was just hoping to double check that even with a very complicated classifier the same probabilistic result held $\endgroup$ – user0 Apr 28 '18 at 23:59
  • $\begingroup$ Yep, still feasible. $\endgroup$ – Ben - Reinstate Monica Apr 29 '18 at 2:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.