I'm not sure if this is hard to answer without the original data, but let's give it a go.

I'm calculating the difference in proportions for two diagnostic methods:

test     0  1
method1 22 30
method2 18 39

Using R and prop.test(c(30, 39), c(52, 57)) gives me a p = 0.3361.

Using R and the infer package "Two categorical (2 level) variables" gives me p = 0.250.

Is such a big difference to be expected when using permutation, or am I doing something wrong?

  • $\begingroup$ What exactly are you doing and why? $\endgroup$ – user2974951 Oct 2 '18 at 9:26
  • $\begingroup$ I'm not sure I understand what you're asking. You mean how and why I'm using that R package? Or the purpose of the comparison? $\endgroup$ – stapperen Oct 2 '18 at 9:31
  • $\begingroup$ How did you get that table? What are those values and methods? What is the purpose of prop.test and what are you doing with the infer package? $\endgroup$ – user2974951 Oct 2 '18 at 9:33
  • $\begingroup$ Alright :) The table is from R table(mydata). The methods are two different biopsy methods for diagnosing cancer, so the outcomes are 0 = no cancer, 1 = cancer. The purpose of prop.test is to test the null hypothesis that there is no difference in detection of cancer with the two methods. I'm testing the same hypothesis using infer with the method described under "Two categorical (2 level) variables" - comparing the observed difference in proportions to the null distribution of 1000 permuted samples. $\endgroup$ – stapperen Oct 2 '18 at 9:40
  • 2
    $\begingroup$ If these are the same cases assessed by the different methods, don't you need a method for paired samples? $\endgroup$ – Björn Oct 2 '18 at 11:11

The difference was due to Yates continuity correction. It's set to default = TRUE in prop.test().

prop.test(c(30, 39), c(52, 57), correct = FALSE) yields a more similar p-value.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.