In Bayesian networks does hard evidence make P(evidence) = 1?

Question

I've been attempting to understand how Bayesian networks work when evidence is applied to them, and in the book I'm currently reading, there are what appear to be contradictory statements, and I don't get what I'm missing.

My assumption before reading this book was that when hard evidence on some variable E (it is definitely one state) is applied, then P(E) = 1.

The book provides an example about a chest clinic, where 5% of all patients are diagnosed with lung cancer, and 50% of all patients are smokers. By analyzing records, they know that the probability that someone was a smoker given that they had lung cancer was 80%. A new patient comes in, and they know the patient is a smoker, so they want to find the probability that the patient will be diagnosed with lung cancer.

The formula they give for this is just Bayes Theorem, where H is "patient has lung cancer" and evidence E is "patient is a smoker": $$ P(H|E) = \dfrac{P(E|H) \times P(H)}{P(E)} $$ Then, plugging in the numbers, they have: $$ P(H|E) = \dfrac{0.8 \times 0.05}{0.5} $$ What I don't understand is why P(E) = .5 in the formula. If you know for a fact that they ARE a smoker, wouldn't that change P(E) = 1? Obviously this would be a problem then if you knew for a fact that they weren't a smoker, because that makes P(E) = 0 and you're dividing by zero, so then what IS the P(E) on bottom and how does observed evidence actually change anything? What does the bottom of the fraction represent?

Later on in the book, when it starts explaining how probabilities in an actual network are calculated, it mentions that if something is known for certain on a node, then the probability for that state of the node is considered to be 1. Why is that not the situation in the chest clinic example?

Is there something fundamental to Bayes theorem I'm just not understanding?
All help is greatly appreciated!

Answer 1

When you say "E" happened with certainly, and are now interested in the probability of lung cancer, that's the same thing as calculating $P(H|E)$. The "given" $E$ part assumes that $E$ happened. There are technical issues that arise when $P(E)=0$ but for the purposes of this discussion that's not an issue.

Take some time to review the definitions of the probability of an event, along with how conditional probability is defined. By assumption the event $E$ is that a given person is a smoker. Without any other information, we have that $P(E)=0.5$. If you ever write $P(whatever)$ you are making a statement about the probability of whatever. What you probably meant to write then is $P(E|E)=1$ which follows immediately from the definition of conditional probability. Given that $E$ occurred, the probability of $E$ is 1.

Now, the observation is: given that you have lung cancer, what's the chance you are a smoker: $P(E|H)=0.8$. With Bayes Rule you can also derive $P(H|E)$, if you also know $P(H)$. More generally, you have that:

$$P(H)=P(H|E)P(E)+P(H|E^c)P(E^c).$$ $$P(E)=P(E|H)P(H)+P(E|H^c)P(H^c).$$

The event of having lung cancer can be broken down into two conditional (mutually exclusive) situations: Either you are a smoker ($E$) or you are not a smoker $(E^c)$.

Answer 2

Let's assume that your thought is true - that, because we are evaluating $P(H|E)$ and we know the individual does have lung cancer, $P(E)=1$. If this is true, this would be true every time we tried to apply Bayes' Theorem and thus it should look like this:

\begin{eqnarray*} P(H|E) &=& \frac{P(E|H)\times P(H)}{P(E)} \\ &=& \frac{P(E|H)\times P(H)}{1} \\ &=& P(E|H)\times P(H) \\ &=& P(E\cap H) \\ \end{eqnarray*}

If that were the case, Bayes' Theorem would be very different and the right-hand side would never have the denominator! In fact, the conditional probability of $H$ occurring given $E$ would just be the joint probability that $E$ and $H$ occur.

The term $P(H|E)$ is the probability that the hypothesis is true given that we know the evidence is true. What we want to do is evaluate the probability that $H$ is true given that, in this case, we know the evidence $E$. That doesn't mean that the probability of $E$ is 1.

If you consider $P(H|E)$ for one person, either $H$ is true or it's not so your probability should always be 0 or 1. However, you're looking at the probability that a randomly selected person has lung cancer given that they're a smoker. That's why you still let $P(E)$ be whatever the probability of smoking is in your population - not that one specific person.

Let's rewrite Bayes' Theorem. \begin{eqnarray*} P(H|E) &=& \frac{P(E|H)\times P(H)}{P(E)} \\ &=& \frac{P(E\cap H)}{P(E)} \\ \end{eqnarray*}

To find the probability of $H$ given $E$, we want to take the probability of $H$ and $E$ occurring together in the population, scaled by the probability that $E$ occurs in the population.