5
$\begingroup$

I've been attempting to understand how Bayesian networks work when evidence is applied to them, and in the book I'm currently reading, there are what appear to be contradictory statements, and I don't get what I'm missing.

My assumption before reading this book was that when hard evidence on some variable E (it is definitely one state) is applied, then P(E) = 1.

The book provides an example about a chest clinic, where 5% of all patients are diagnosed with lung cancer, and 50% of all patients are smokers. By analyzing records, they know that the probability that someone was a smoker given that they had lung cancer was 80%. A new patient comes in, and they know the patient is a smoker, so they want to find the probability that the patient will be diagnosed with lung cancer.

The formula they give for this is just Bayes Theorem, where H is "patient has lung cancer" and evidence E is "patient is a smoker": $$ P(H|E) = \dfrac{P(E|H) \times P(H)}{P(E)} $$ Then, plugging in the numbers, they have: $$ P(H|E) = \dfrac{0.8 \times 0.05}{0.5} $$ What I don't understand is why P(E) = .5 in the formula. If you know for a fact that they ARE a smoker, wouldn't that change P(E) = 1? Obviously this would be a problem then if you knew for a fact that they weren't a smoker, because that makes P(E) = 0 and you're dividing by zero, so then what IS the P(E) on bottom and how does observed evidence actually change anything? What does the bottom of the fraction represent?

Later on in the book, when it starts explaining how probabilities in an actual network are calculated, it mentions that if something is known for certain on a node, then the probability for that state of the node is considered to be 1. Why is that not the situation in the chest clinic example?

Is there something fundamental to Bayes theorem I'm just not understanding?
All help is greatly appreciated!

$\endgroup$
  • $\begingroup$ P(E) = 0.5 is a non-committing prior basically stating that a priori the new person who shows up has 50/50 chances being smoker. In other words, they don't know is the person a smoker or not - no preference, even slightest one, to one or another. If something about the person is observed (e.g. coughing, yellow skin, unhealthy look), this is not P(E) anymore but some P(E|evidence). This is why it is often said that previous study posteriors are priors to the current study. $\endgroup$ – Vladislavs Dovgalecs Jun 17 '16 at 21:11
3
$\begingroup$

When you say "E" happened with certainly, and are now interested in the probability of lung cancer, that's the same thing as calculating $P(H|E)$. The "given" $E$ part assumes that $E$ happened. There are technical issues that arise when $P(E)=0$ but for the purposes of this discussion that's not an issue.

Take some time to review the definitions of the probability of an event, along with how conditional probability is defined. By assumption the event $E$ is that a given person is a smoker. Without any other information, we have that $P(E)=0.5$. If you ever write $P(whatever)$ you are making a statement about the probability of whatever. What you probably meant to write then is $P(E|E)=1$ which follows immediately from the definition of conditional probability. Given that $E$ occurred, the probability of $E$ is 1.

Now, the observation is: given that you have lung cancer, what's the chance you are a smoker: $P(E|H)=0.8$. With Bayes Rule you can also derive $P(H|E)$, if you also know $P(H)$. More generally, you have that:

$$P(H)=P(H|E)P(E)+P(H|E^c)P(E^c).$$ $$P(E)=P(E|H)P(H)+P(E|H^c)P(H^c).$$

The event of having lung cancer can be broken down into two conditional (mutually exclusive) situations: Either you are a smoker ($E$) or you are not a smoker $(E^c)$.

$\endgroup$
  • $\begingroup$ "E part assumes that E happened" be careful, in Bayesian setting there is not necessarily time factor. In other words P(H|E) can assume that E first happened, then H happened but this is generally not true always. $\endgroup$ – Vladislavs Dovgalecs Jun 17 '16 at 21:18
  • 1
    $\begingroup$ @xeon: This has nothing to do with time, which is something that is tacked into the philosophy of applied probability. $P(E|E)$ as written is with respect to an event $E$ that is perfectly well defined in some abstract space. If you want to introduce time then write $P(E(t)|E(s))$. $\endgroup$ – Alex R. Jun 17 '16 at 21:33
  • $\begingroup$ Sure. I just wanted to comment to other readers to be aware of this. Not more, not less. $\endgroup$ – Vladislavs Dovgalecs Jun 17 '16 at 21:58
1
$\begingroup$

Let's assume that your thought is true - that, because we are evaluating $P(H|E)$ and we know the individual does have lung cancer, $P(E)=1$. If this is true, this would be true every time we tried to apply Bayes' Theorem and thus it should look like this:

\begin{eqnarray*} P(H|E) &=& \frac{P(E|H)\times P(H)}{P(E)} \\ &=& \frac{P(E|H)\times P(H)}{1} \\ &=& P(E|H)\times P(H) \\ &=& P(E\cap H) \\ \end{eqnarray*}

If that were the case, Bayes' Theorem would be very different and the right-hand side would never have the denominator! In fact, the conditional probability of $H$ occurring given $E$ would just be the joint probability that $E$ and $H$ occur.

The term $P(H|E)$ is the probability that the hypothesis is true given that we know the evidence is true. What we want to do is evaluate the probability that $H$ is true given that, in this case, we know the evidence $E$. That doesn't mean that the probability of $E$ is 1.

If you consider $P(H|E)$ for one person, either $H$ is true or it's not so your probability should always be 0 or 1. However, you're looking at the probability that a randomly selected person has lung cancer given that they're a smoker. That's why you still let $P(E)$ be whatever the probability of smoking is in your population - not that one specific person.

Let's rewrite Bayes' Theorem. \begin{eqnarray*} P(H|E) &=& \frac{P(E|H)\times P(H)}{P(E)} \\ &=& \frac{P(E\cap H)}{P(E)} \\ \end{eqnarray*}

To find the probability of $H$ given $E$, we want to take the probability of $H$ and $E$ occurring together in the population, scaled by the probability that $E$ occurs in the population.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.