First off let me say that I had one stats course in engineering school 38 years ago. So I'm flying blind here.

I've got the results of what are essentially 18 separate diagnostic tests for a disease. Each test is binary -- yes/no, with no threshold that can be adjusted to "tune" the test. For each test I have what is ostensibly valid data on true/false positives/negatives when compared to the "gold standard", yielding specificity and sensitivity numbers (and anything else you can derive from that data).

Of course, no single test has sufficient specificity/sensitivity to be used alone, and when you "eyeball" the results of all tests there's frequently no obvious trend.

I'm wondering what is the best way to combine these numbers in a way that will yield a final score that is (hopefully) more reliable than any single test. I've so far come up with the technique of combining of the specificities of TRUE tests using

spec_combined = 1 - (1 - spec_1) * (1 - spec_2) * ... (1 - spec_N)

and combining sensitivities of the FALSE tests the same way. The ratio

(1 - sens_combined) / (1 - spec_combined) 

then seems to yield a reasonably good "final score", with a value over 10 or so being a reliable TRUE and a value under 0.1 or so being a reliable FALSE.

But this scheme lacks any true rigor, and for some combinations of test results it seems to produce an answer that is counter-intuitive.

Is there a better way to combine the test results of multiple tests, given their specificities and sensitivities? (Some tests have a specificity of 85 and sensitivity of 15, other tests are just the opposite.)

##OK, my head hurts!

Let's say I've got tests 1-4 with sensitivities/specificities (in %):

1. 65/50
2. 25/70
3. 30/60
4. 85/35

Tests 1 and 2 are positive, 3 and 4 negative.

The putative probability that 1 is a false positive would be (1 - 0.5), and for 2 (1 - 0.7), so the probability that both are false positives would be 0.5 x 0.3 = 0.15.

The putative probability that 3 and 4 are false negatives would be (1 - 0.3) and (1 - 0.85) or 0.7 x 0.15 = 0.105.

(We'll ignore for the moment the fact that the numbers don't add up.)

But the presumed probabilities that 1 and 2 are true positives are 0.65 and 0.25 = 0.1625, while the presumed probabilities that 3 and 4 are true negatives are 0.6 and 0.35 = 0.21.

Now we can ask two questions:

1. Why don't the numbers add up (or even come close). (The sens/spec numbers I used are from "real life".)
2. How should I decide which hypothesis is (most likely) true (in this example it seems to be "negative" for both calcs, but I'm not sure that's always the case), and what can I use for a "figure of merit" to decide if the result is "significant"?

##More info

This is an attempt to refine and extend an existing "weighting" scheme that is entirely "artistic" in nature (ie, just pulled out of someone's a**). The current scheme is basically on the lines of "If any two of the first three are positive, and if two of the next four, and either of the next two, then assume positive." (That's a somewhat simplified example, of course.) The available statistics don't support that weighting scheme -- even with a crude weighting algorithm based on the measured stats I come up with significantly different answers. But, absent a rigorous way of evaluating the stats I have no credibility.

Also, the current scheme only decides positive/negative, and I need to create a (statistically valid) "ambiguous" case in the middle, so some figure of merit is needed.