My question is not literally the formula for measuring sensitivity and specificity of a test. Instead, I am asking how do different medical tests decide what constitutes a reported TP/TN/FP/FN, and how is that related to the structure of their trial?
For example, I develop a new medical test for condition X which occurs in 10 out of every 100 individuals in the broader population. I claim my test has 95% sensitivity and 99% specificity based on trials. But wouldn't the structure of the trial change the observed accuracy? I could structure it something like:
- Test 1,000 random population individuals
- Test 1,000 individuals but over-sample those with the condition (maybe 50/50)
I would think scenario 1 would experimentally show my test being less effictive while scenario 2 amplifies it's power. Does the proportion of classes in a study affect how I should infer test accuracy?
Related question would be how do medical sensitivity/specificity numbers change depending on other patient observations? For example:
- An a patient walks into a doctor's office and a doctor randomly orders a test with 95%/99% accuracy
- A patient showing symptoms consistent with the disease and the doctor orders the same test