1
$\begingroup$

I am trying to figure out the stats in a paper Assessment of Deep Generative Models for High-Resolution Synthetic Retinal Image Generation of Age-Related Macular Degeneration

Researchers gave a specialist a set of images, told they the set consists of real images and synthetic images (simply, some kind of fake images), asked they distinguish real ones from the fake.

Assume there are 100 real images provided in the test and the specialist tagged 65 of them as "real" and 35 of them as "fake".

So, the TP = 65, FP = 35, ${\displaystyle \mathrm {PPV} ={\frac {\mathrm {TP} }{\mathrm {TP} +\mathrm {FP} }} = \frac{65}{100}}$

Is my understanding about this correct?

$\endgroup$
2
$\begingroup$

No, that is not correct.

"Positive" and "negative" refer to the ground truth, and "false" and "true" to whether the classification accords with the ground truth. In the present case, real images are "positive", and fake ones are "negative". Thus, $\text{FP}$ is the number of images the specialist tags as real, but are actually fake.

The number 35 you have are the number that the specialist tagged as fake ("negative") but which aren't, so this is the number $\text{FN}$ of false negatives, not false positives.

You can't calculate the $\text{PPV}$ from the data you present here, since you can't calculate $\text{FP}$.

More information can be found at the Wikipedia page on sensitivity and specificity.

$\endgroup$
0
$\begingroup$

Let's start off with writing down the mathematical definitions for the statistical measures of interest to us. From there we can work our way up to an example.

Sensitivity measures the percentage of true positives that are correctly identified as being positive.

formula

Specificity measures the percentage of true negatives that are correctly identified as being negative.

formula

The positive predictive value (PPV) or P(D|+) is the probability that the subject has the disease given that the test is positive. To calculate PPV, we will need the probability of a positive test result given disease P(+|D) and its complement, no disease P(+|DC)

formula

Diagnostic Likelihood Ratios measure the post-test odds (after test) compared to pre-test odds (before test) of either having the disease or not.

formula

formula

Ok, so we've got the definitions down, let's work through an example.

At the moment there are approximately 8,000 people infected with COVID-19
living in the city of San Francisco. With a population of ~ 1 million people,
calculate both the DLR+ and PPV. How can we interpret these results? Assume a
pharmaceutical company has developed an antibody (Ab) test with a sensitivity 
of 93% percent and a specificity 99% percent. `

We first need to calculate the prevalence of disease p(D) = 8,000/1,000,000 = .008 and using the p(D) we can calculate the probability of no disease p(DC) = (1 - p(D)) = .992

Now from the PPV formula, we need to know the probability of a person testing positive for COVID-19 even though they actually don't have it p(+|DC) = 1 - p(-|DC) = 1 - specificity = .01.

PPV = (.93*.008) / (.93*.008 + .01*.992) ~ .43

DLR+ = Sensitivity / (1 - Specificity) = .93 / (1 - .99) = 93

So what do these numbers actually mean? How can we interpret DLR+ and PPV?

  1. The PPV calculation suggests that after testing positive for the disease, we actually only have a ~ 43% of actually having the disease. Interesting right.

  2. The DLR+ tells us that a positive test result increases the post-test odds of disease by 93X compared to pre-test odds. The hypothesis of disease is 93X more supported by data than a hypothesis of no disease.

Hopes this helps. A final note: In reality, there are a lot of asymptomatic people with COVID-19 who don't get tested but are nonetheless positive (more true positives). What changes in the problem above?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.