Interpretation of Bayes Theorem applied to positive mammography results

Question

I'm trying to wrap my head around the result of Bayes Theorem applied to the classic mammogram example, with the twist of the mammogram being perfect.

That is,

Incidence of cancer: $.01$

Probability of a positive mammogram, given the patient has cancer: $1$

Probability of a positive mammogram, given the patient does not have cancer: $.01$

By Bayes:

P(cancer | mammogram+) = $\dfrac {1 \cdot .01}{(1 \cdot .01) + (.091 \cdot .99)}$

$ = .5025$

So, if a random person from the population takes the mammogram, and obtains a positive result, there is a 50% chance they have cancer? I'm failing to intuitively understand how the tiny 1% chance of a false positive in 1% of the population can trigger a 50% result. Logically, I would think a perfectly true positive mammogram with a tiny false positive rate would be much more accurate.

Answer 1

I will answer this question both from a medical and a statistics standpoint. It has received a lot of attention in the lay press, particularly after the best-seller The Signal and the Noise by Nate Silver, as well as a number of articles in publications such as The New York Times explaining the concept. So I'm very glad that @user2666425 opened this topic on CV.

First off, let me please clarify that the $p\,(+|C) = 1$ is not accurate. I can tell you that this figure would be a dream come true. Unfortunately there are a lot of false negative mammograms, particularly in women with dense breast tissue. The estimated figure can be $20\%$ or higher, depending on whether you lump all different types of breast cancers into one (invasive v DCIS), and other factors. This is the reason why other modalities based on sonographic or MRI technology are also applied. A difference between $0.8$ and $1$ is critical in a screening test.

Bayes theorem tells us that $\small p(C|+) = \large \frac{p(+|C)}{p(+)}\small* p(C)$, and has recently gotten a lot attention as it relates to mammography in younger, low risk women. I realize this is not exactly what you are asking, which I address in the final paragraphs, but it is the most debated topic. Here is a taste of the issues:

1. The prior (or probability of having cancer based on prevalence) in younger patients, say from 40 - 50 years of age is rather small. According to the NCI it would can round it up at $\sim 1.5\%$ (see table below). This relatively low pre-test probability in itself reduces the post-test conditional probability of having cancer given that the mammogram was positive, regardless of the likelihood or data collected.

2. The probability of a false positive becomes a very significant issue on a screening procedure that will be applied to thousands and thousands of a priori healthy women. So, although the false positive rate of $7 - 10\%$ (which is much higher if you focus on the cumulative risk) may not sound so bad, it is actually an issue of colossal psychological and economical costs, particularly given the low pre-test probability in younger, low-risk patients. Your figure of $1\%$ is widely off the mark - the reality is that "scares" are incredibly common due to many factors, including the medicolegal concerns.

So, recalculating and very importantly, for younger women without risk factors:

$p(C|+) = \frac{p(+|C)}{p(+)}\small* p(C) =$

$= \frac{p(+|C)}{p(+|C)\,*\,p(C)\, +\, p(+|\bar C)\,*\,p(\bar C)}\small* p(C) = \large \frac{0.8}{0.8*0.015\, +\, 0.07*0.985}\small*\, 0.015 = 0.148$.

The probability of having cancer when a screening mammogram has been read as positive can be as low as $15\%$ in young, low-risk women. As an aside, mammographic readings come with an indirect estimate of the confidence in the diagnosis the radiologist has (it is called BI-RADS), and this Bayesian analysis would change radically as we progress from a BI-RADS 3 to a BI-RADS 5 - all of them "positive" tests in the broadest sense.

This figure can logically be changed depending on what estimates you consider in your calculation, but the truth is that the recommendations for the starting age to enter a screening mammography program have recently been pushed up from age $40$ to $45$.

In older women the prevalence (and hence the pre-test probability) increases linearly with age. According to the current report, the risk that a woman will be diagnosed with breast cancer during the next 10 years, starting at the following ages, is as follows:

Age 30 . . . . . . 0.44 percent (or 1 in 227)
Age 40 . . . . . . 1.47 percent (or 1 in 68)
Age 50 . . . . . . 2.38 percent (or 1 in 42)
Age 60 . . . . . . 3.56 percent (or 1 in 28)
Age 70 . . . . . . 3.82 percent (or 1 in 26)

This results in a life-time cumulative risk of approximately $10\%$:

The calculation in older women with a prevalence of $4\%$ would be:

$p(C|+)=\large \frac{0.8}{0.8*0.04\, +\, 0.07*0.96}\small*\, 0.04 = 0.32 \sim 32\%$ lower than you calculated.

I can't overemphasize how many "scares" there are even in older populations. As a screening procedure a mammogram is simply the first step so it makes sense for the positive mammogram to be basically interpreted as there is a possibility that the patient has breast cancer, warranting further work-up with ultrasound, additional (diagnostic) mammographic testing, follow-up mammograms, MRI or biopsy. If the $p(C|+)$ was very high we wouldn't be dealing with a screening test it would be a diagnostic test, such as a biopsy.

Specific answer to your question:

It is the "scares", the $p(+|\bar C)$ of $7-10\%$, and not $1\%$ as in the OP, in combination with a relative low prevalence of disease (low pre-test probability or high $p(\bar C)$) especially in younger women, that accounts for this lower post-test probability across ages. Notice that this "false alarm rate" is multiplied by the much larger proportion of cases without cancer (compared with patients with cancer) in the denominator, not the "the tiny 1% chance of a false positive in 1% of the population" you mention. I believe this is the answer to your question. To emphasize, although this would be unacceptable in a diagnostic test, it is still worthwhile in a screening procedure.

Intuition issue: @Juho Kokkala brought up the issue that the OP was asking about the intuition. I thought it was implied in the calculations and the closing paragraphs, but fair enough... This is how I would explain it to a friend... Let's pretend we are going hunting for meteor fragments with a metal detector in Winslow, Arizona. Right here:

Image from meteorcrater.com

... and the metal detector goes off. Well, if you said that chances are that it is from a coin a tourist dropped off, you'd probably be right. But you get the gist: if the place hadn't been so thoroughly screened, it would be much more likely that a beep from the detector on a place like this came from a fragment of meteor than if we were on the streets of NYC.

What we are doing with mammography is going to a healthy population, looking for a silent disease that if not caught early can be lethal. Fortunately, the prevalence (although very high compared with other less curable cancers) is low enough that the probability of randomly encountering cancer is low, even if the results are "positive", and especially in young women.

On the other hand, if there were no false positives, i.e. ($p(\bar C|+)=0$,

$\frac{p(+|C)}{p(+|C)\,*\,p(C)\, +\, p(+|\bar C)\,*\,p(\bar C)}\small* p(C) = \frac{p(+|C)}{p(+|C)\,*\,p(C)}\small* p(C) = 1$, much as the probability of having hit a meteor fragment if our metal detector went off would be $100\%$ independent of the area we happened to be exploring if instead of a regular metal detector we were using a perfectly accurate instrument to detect outer-space amino acids in the meteor fragment (made-up example). It would still be more likely to find a fragment in the Arizona desert than in New York City, but if the detector happened to beep, we'd know we had found a meteor.

Since we never have a perfectly accurate measuring device or system, the fraction $\frac{\text{likelihood}}{\text{unconditional p(+)}}=\frac{p(+|C)}{p(+|C)\,*\,p(C)\, +\, p(+|\bar C)\,*\,p(\bar C)}$ will be $<1$, and the more imperfect it is, the lesser the fraction of the $p(C)$, or prior, that will be "passed on" to the LHS of the equation as the posterior. If we settle on a particular type of detector, the likelihood fraction will act as constant in a linear equation of the form, $\text{posterior} = \alpha * \text{prior}$, where the $\text{posterior} < \text{prior}$, and the smaller the prior, the linearly smaller will the posterior be. This is referred to as the dependence on prevalence of the positive predictive value (PPV): probability that subjects with a positive screening test truly have the disease.

Answer 2

A key problem with mammography that hasn't been adequately addressed in the discourse is the faulty definition of "positive". This is described in the Diagnosis chapter in http://biostat.mc.vanderbilt.edu/ClinStat - see the link for Biostatistics in Biomedical Research there.

One of the most widely used diagnostic coding systems in mammography is the BI-RADS score, and a score of 4 is a frequent "positive" result. The definition of category 4 is "Not characteristic of breast cancer, but reasonable probability of being malignant (3 to 94%); biopsy should be considered." With a risk range that goes all the way from 0.03 to 0.94 for one category, i.e., incredible heterogeneity in what "positive" really means, it is no wonder that we have a mess on our hands.

It is also a sign of unclear thinking that the BI-RADS system has no category for someone with an estimated risk of 0.945.

As Nate Silver so eloquently argues in The Signal and the Noise, if we were to think probabilistically we would make better decisions all around. Removing terms such as "positive" and "negative" for medical tests would remove false positives and false negatives and convey uncertainty (and justification for more tests before making a diagnosis) optimally.

Answer 3

There is a nice discussion of this in the book Calculated Risks

Much of the book is about finding clearer ways of talking about, and thinking about, probability and risk. An example:

The probability that a woman of age 40 has breast cancer is about 1 percent. If she has breast cancer, the probability that she will test positive on a screening mammogram is about 90 percent. If she does not have breast cancer, the probability that she will nevertheless test positive is 9 percent. What are the chances that a woman who tests positive actually has breast cancer?

This is the way the book presents the solution, using 'natural frequencies'. Consider 10,000 women, 1% have cancer so that is 100 women. Of these, 90% will return positive tests (i.e. 90 women with cancer will test positive). Of the 9900 without cancer 9% will return positive test or 891 women. So there are 891 + 90 = 981 women with positive tests of which 90 have cancer. So the chance that a woman with a positive test has cancer is 90/981 = 0.092

If 100% of woman with cancer test positive that just changes the numbers a bit to 100/(100 + 891) = 0.1

Answer 4

Perhaps this line of thinking is correct?:

For any random person, there is a 1% chance that they have cancer, and so there is then a $.01 * 1$ chance that the mammogram of a random person will be positive. If they do not have cancer, there is a 1% chance that the mammogram will be positive.

So it is intuitively close to a coin flip for a random person. I'm not sure how to explain the additional $0.0025$ in favor of cancer given a positive mammogram.

Answer 5

Here's an oversimplified but intuitive way to look at it. Consider 100 people. One has cancer and will test positive. Of the 99 who don't, one of them will get a false positive test. So of the two positives, one will have cancer and one won't.