# Applying Bayes's Theorem When Evidence is Uncertain

My limited understanding of Bayes's theorem,

$$P(H|E)=\frac{P(E|H)P(H)}{P(E)}$$

is that—even though one of it's terms is $P(E)$—it's applied when $E$ is definitely known to be true. (You went to the doctor and definitely got a positive test result.)

How can I apply the theorem when $E$ might be true with some known probability?

More detail: I have a machine-learning model that, evaluating $H$ and $E$ individually, estimates how probable each is. $H$ and $E$ may be are related, though, so I'd like to adjust these probabilities based on what the model said about the other. ($H$ could be true even if $E$ is not true—but $E$ being true makes it more likely $H$ is true.)

A concrete example adapted from Wikipedia's:

Suppose that a test for using a particular drug is 99% sensitive and 99% specific. That is, the test will produce 99% true positive results for drug users and 99% true negative results for non-drug users. Suppose that 0.5% of people are users of the drug.

What is the probability that a randomly selected individual for whom it is 70% certain (somehow) that he got a positive result is a user of the drug? (This does not mean that's there's a 30% chance he got a negative result, but a 30% chance that he got no result.)

• Interesting question. I don't have the answer, but this seems relevant pdfs.semanticscholar.org/13f5/… – Michael Lew - reinstate Monica May 9 '18 at 0:00
• How can you adjust the probabilities if you don't even know if $H$ and $E$ are related in the first place? – jbowman May 9 '18 at 0:43
• @jbowman poor wording. They are related. (Edit: on second, thought, I haven't completely worked on the conditional probability, but is it true that Bayes's theorem require that they be definitely related?) – Randoms May 9 '18 at 0:48
• I'm not entirely sure what you mean - in general $P(E) = \int P(E|H)p(H)\mathrm{d}H$ and you use this to update the probability distribution of $H$. So you are effectively averaging over the uncertainty of the value of $H$. Can you give a concrete example of a situation where this arises? That might make it easier to help. – Maurits M May 9 '18 at 7:52
• @MauritsM Does that equation have a name? Please see the example I added. – Randoms May 9 '18 at 21:54

Let $E$ be an event that occurs with probability $0 \leqslant \phi \leqslant 1$. Assuming that $H \perp \phi | E$ (which just means that your probability $\phi$ has no further effect on the calculations), you have:

\begin{aligned} \mathbb{P}(H|\phi) &= \mathbb{P}(H,E|\phi) + \mathbb{P}(H,\bar{E}|\phi) \\[6pt] &= \mathbb{P}(H|E,\phi) \mathbb{P}(E|\phi) + \mathbb{P}(H|\bar{E},\phi) \mathbb{P}(\bar{E}|\phi) \\[6pt] &= \phi \cdot \mathbb{P}(H|E) + (1-\phi) \cdot \mathbb{P}(H|\bar{E}) \\[6pt] &= \phi \cdot \frac{\mathbb{P}(E|H) \mathbb{P}(H)}{\mathbb{P}(E)} + (1-\phi) \cdot \frac{\mathbb{P}(\bar{E}|H) \mathbb{P}(H)}{\mathbb{P}(\bar{E})} \\[6pt] &= \phi \cdot \frac{\mathbb{P}(E|H) \mathbb{P}(H)}{\mathbb{P}(E)} + (1-\phi) \cdot \frac{(1-\mathbb{P}(E|H)) \mathbb{P}(H)}{1-\mathbb{P}(E)}. \\[6pt] \end{aligned}

This gives you an extended version of Bayes' rule for dealing with an uncertain event $E$. As you can see, this rule merely says that the conditional probability of $H$ given $\phi$ can be considered as a weighted average of the Bayes' rule expressions for the two possible outcomes of $E$, weighted by their probabilities.

• What does $\bar{E}$ mean here? – Randoms May 10 '18 at 0:09
• That means not-$E$ (i.e., the event $E$ does not occur). I have added an additional line to the equation to show the result just in terms of $E$. – Reinstate Monica May 10 '18 at 0:10
• Thanks for clarifying. I'm wondering, though, does this assume $P(\bar{E}|\phi) = (1 - \phi)$? In that $(1 - \phi)$ portion of outcomes, we don't necessarily have $\bar{E}$—we just don't know whether we have $E$ or $\bar{E}$. – Randoms May 10 '18 at 2:24
• We have assumed that $\mathbb{P}(E|\phi) = \phi$, so we must also have $\mathbb{P}(\bar{E}|\phi) = 1-\mathbb{P}(E|\phi) = 1-\phi$. That is not an assumption; it is a consequence of our specification of the probability that $E$ occurs. – Reinstate Monica May 10 '18 at 8:36

Well, first of all, "evidence" is generally taken to be, by definition, that which you are certain of. If your doctor comes to you and says "We got your results mixed up with five other people's, and got five positive results", then you apply Bayes' theorem to P(E|H), where H = you have the disease and E = "We got your results mixed up with five other people's, and got five positive results". If you don't know enough about E to calculate P(E|H), then you can't apply Bayes' Theorem.

With your example, consider 1 000 000 people. 5 000 are drug users. Of those, 995 will test positive, and 5 negative. There are 995 000 non-drug users, of which 9950 will test positive and 985050 will test negative. So, out of those who test positive, 9.09% are drug users. But we don't have that the person in question tested positive, we have that they "are 70% certain to have tested positive". So now we have to find out what that population is made up of. One issue is presumably, 70% of people who are 70% certain to have tested positive tested positive. But what of the other 30%? Did the test negative? Or weren't tested at all? Let's say they tested negative. We have that 9.09% of positive are drug users, and they make up 70%. So this gives 6.36% drug users. We also have that those a negligible percentage that tested negative drug users. We add those numbers together and get that 6.36% of this set are drug users.

So to recap:

percentage positive * P(drug user|positive) + percentage negative * P(drug user|negative)

which is

.70(true positive/(true positive + false positive))+.30(false negative/(false negative + true negative))

However, this is based on certain independence assumptions. Let's call these people that are 70% certain "reported". My calculation assumes that false positives are just as likely to be reported as true positives. But it is very likely that that will not be true.