How to tell a method is improving the detection of a disease? We have two methods, A and B, for detecting a disease. Out of a 1000 samples with unknown prevalence, A detects 100 samples (10%). On the remaining 900 negatives, we apply the additional method B, and detect a further 20. As far as we can tell, all these are true positives.
We don't really know what the specificity / sensitivity of both methods is, because it depends on the prevalence, and we don't know that.
However, we would like to have an idea how much we can trust the observed increase by 20 in future applications. Is it real? Does the method B significantly improve the detection? Will it be a good idea to add B to the process? If we detected 100 samples in the first method, and 2 in the second, we would not trust it so much, would we? 
We could produce a contingency table
           method A    method A+B
 positive      100         120
 negative      900         880

and test it with Chi^2^ or smth., but I think that this would be incorrect - the data in the second column include the data from the first column. Also, we are not interested in comparing the methods, after all, the method B is a "second line" and is not directly tested on samples that can be detected with A.
I am at loss as how to tackle this question. The problem is real, the disease deadly and the numbers are close enough to be real. I had no influence on the study design, we have what we have, but it is important enough to try to find an answer to the above question.
 A: So it seems that with the current structure, we don't have any information about the negatives. If I understand the problem correctly, what we have is that:


*

*Method A had 100 true positives, 0 false positives, at least 20 false negatives and no information about the true negatives negatives in 1000 cases tested. 

*Method B had 20 true positives, 0 false positives and no information about the negatives in 900 cases tested.

*Method A + B had 120 true positives, 0 false positives and no information about the negatives in 1000 cases tested.


That being said, from these stats you can calculate some relevant diagnostic test metrics however, without any other information (e.g: prevalence, additional "cost" of applying method B, classification of the negatives etc) this info will not add any value.
Finally, as Stephan said, if the disease is deadly then the cost of a false negative is "extremely high" and without any additional information we can't really infer much and I don't see any justification why not including method B.
PS: Can we at least test Method B alone in the 1000 cases? Maybe method A is not needed at all.
A: We're having to take a lot on good faith here, but accepting your restrictions then I would suggest that this is a valid scenario for deploying McNemar's test. 
Whether this gives the information you want is another matter but if I'm interpreting your post correctly it would. It does not test accuracy, so will not allow you to use it to claim accuracy, just whether the difference caused by B is significant. This test is designed to assess if there is significant imbalance in how two tests disagree or if there is significant movement of cases between two conditions. It ignores the main diagonal and looks only at the off diagonal elements, where the conditions are mismatched.
First you need the contingency table updated
+-------+-----+------+
|       | A:P | A:N  |
+-------+-----+------+
| A+B:P | 100 |   20 |
| A+B:N |   0 |  880 |   
+-------+-----+------+

In this case you are comparing agreement /disagreement between two related protocols rather than independent tests. My understanding of the logic of A+B is that if A then +, else if B +else negative. I believe this is still valid for McNemar's as it is intended for comparing conditions that are related.
Rather than go into the details here I'll link to various resources :
https://en.m.wikipedia.org/wiki/McNemar%27s_test
https://statistics.laerd.com/spss-tutorials/mcnemars-test-using-spss-statistics.php
What is the difference between McNemar's test and the chi-squared test, and how do you know when to use each?
www.researchgate.net/post/How_to_use_McNemars_test_to_compare_accuracy_of_classifications
