# How are specificity and sensitivity for medical tests measured?

My question is not literally the formula for measuring sensitivity and specificity of a test. Instead, I am asking how do different medical tests decide what constitutes a reported TP/TN/FP/FN, and how is that related to the structure of their trial?

For example, I develop a new medical test for condition X which occurs in 10 out of every 100 individuals in the broader population. I claim my test has 95% sensitivity and 99% specificity based on trials. But wouldn't the structure of the trial change the observed accuracy? I could structure it something like:

1. Test 1,000 random population individuals
2. Test 1,000 individuals but over-sample those with the condition (maybe 50/50)

I would think scenario 1 would experimentally show my test being less effictive while scenario 2 amplifies it's power. Does the proportion of classes in a study affect how I should infer test accuracy?

Related question would be how do medical sensitivity/specificity numbers change depending on other patient observations? For example:

1. An a patient walks into a doctor's office and a doctor randomly orders a test with 95%/99% accuracy
2. A patient showing symptoms consistent with the disease and the doctor orders the same test

The sensitivity is calculated purely from the true cases so the number of the true non-cases is not relevant and vice versa. However if the decision is being made by a human diagnostician and s/he knows the prevalence of the condition it may affect his/her criterion. The positive and negative predictive values are of course strongly affected by the prevalence.

You may be interested in

@ARTICLE{lijmer99,
author = {Lijmer, J G and Mol, B W and Heisterkamp, S and
Bonsel, G J and Prins, M H and van der Meulen, J H P
and Bossuyt, P M M},
year = {1999},
title = {Empirical evidence of design-related bias in
studies of diagnostic tests},
journal = {Journal of the American Medical Association},
volume = 282,
pages = {1061--1066}
}


which discusses a number of ways in which studies of diagnostic tests can be subject to bias induced by poor design.