# Comparing Proportion of Positive Tests

A condition or disease (D) is measured using two different methods (A and B) in a sample of 1,000 individuals from a population. Using method A, the percentage of positive cases is 25%, whereas method B yields 30%.

Is there a test of statistical significance I can apply to compare method A and method B? How can we test for whether these different percentages are wide enough to conclude that we are dealing with two methods of testing for D that are essentially different beyond statistical noise?

I was assuming that it would amount to testing two different proportions, but when I plug the data and analyze it with R I realize that the categorical options for method A and B are identical (i.e. positive and negative), and the comparison of proportions doesn't make sense.

It's a basic question, but I'm stuck... Here is the data:

  str(dat)
'data.frame':  1000 obs. of  2 variables:
$Method.A: Factor w/ 2 levels "neg","pos": 1 1 1 1 2 2 1 1 1 1 ...$ Method.B: Factor w/ 2 levels "neg","pos": 1 1 1 1 2 2 1 1 1 1 ...

summary(dat)
Method.A  Method.B
neg:574   neg:488
pos:426   pos:512

countdf
Method.A Method.B Freq
1      neg      neg  302
2      pos      neg  186
3      neg      pos  272
4      pos      pos  240


Clearly the methods are different, with Method B tending to show more positives (True or False), but if I try to run a Chi-square test it will show a statistically significant p value, implying correlation between both Methods, which of course it's there because both methods attempt to detect the disease, and neither one is awful.

What I need is to show that they are different in the way they go about detecting D, because they can't possibly be equivalent when the positives for Method B are 512 while Method A only flags 426 subjects as positive.

Just for completeness:

Percentage table:
Method.B
Method.A  neg  pos Total Count
neg 52.6 47.4   100   574
pos 43.7 56.3   100   426

2-sample test for equality of proportions without continuity
correction

data:  .Table
X-squared = 7.8415, df = 1, p-value = 0.005106
alternative hypothesis: two.sided
95 percent confidence interval:
0.02716934 0.15185603
sample estimates:
prop 1    prop 2
0.5261324 0.4366197


Following the lead of another thread of discussion vis-a-vis Agresti's matched observations (Testing paired frequencies for independence), I went one step further in my answer, and came up with the following results.

Of note, I realize that I am treating these values as ordinal, and methodologically it is a stretch, but I'm getting the same result applying a poisson glm.

two_by_two<-matrix(c(240,186,272,302),nrow=2,byrow=T)
dimnames(two_by_two) <- list(c("pos","neg"),c("pos","neg"))
names(dimnames(two_by_two)) <-c("Method.A","Method.B")
require(vcd)
library(gnm)
tab <- data.frame(counts=c(two_by_two), Method.A=gl(2,1,4,labels=c("neg","pos")),
Method.B=gl(2,2,4,labels=c("neg","pos")))