Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I am calculating two different $p$ values for the McNemar test and what I believe is the corresponding exact test, even for combined counts larger than 25.

Take, for example the contingency table

                   Posttest
Pretest      Outcome A   Outcome B
Outcome A       xx          50
Outcome B       65          xx

The chi-square statistic is $$\chi^2=\frac{(50-65)^2}{50+65}=1.96,$$ with $$p=0.1619. \tag{R}$$

According to Sheskin (2011, Test 20, VI.3, pg 844), the exact test for these situations is essentially a binomial sign test (for a single sample) with parameter $\pi=0.5$ and the two counts equal to the two the cells of interest in the contingency table. My understanding of this is that I'm calculating the probability that, out of $50+65=115$ trials with probability of success 0.5, at least 65 of them are successes. My attempt to calculate this for the particular situation above is: $$ p = \sum_{i=65}^{115} \begin{pmatrix} 115 \\ i \end{pmatrix} 0.5^{i}\ 0.5^{115-i} = 0.0957 \tag{Mathematica} $$

Why do these two $p$-values differ so much? Am I making a simple mistake?

share|improve this question
    
I've just one again checked it in SPSS. McNemar = Sign test for dichotomous data (asymptotic = asymptotic, exact = exact). –  ttnphns Jun 5 at 6:30

2 Answers 2

up vote 4 down vote accepted

Two points:

1. Continuity Correction

McNemar's test uses discrete data (counts of discordant pairs) to produce a $\chi^{2}$ test statistic with one degree of freedom. However, the $\chi^{2}$ distribution is not discrete, but continuous. Hence, the need for a continuity correction.

$\chi^{2}=\frac{\left(\left|50-65\right|-1\right)^{2}}{\left(50+65\right)} = 1.781818$

$p = \text{P}\left(X^{2}_{\text{df}=1} \ge \chi^{2}\right) = 0.182$

2. Two-sided Test

$0.0957 \times 2 = 0.191 \approx 0.182$

Although, even without the continuity correction $0.191 \approx 0.162$ (kinda).

share|improve this answer
    
Thanks! I thought something was fishy when I was leaving out the sum from 0 to 50. And I think I have more respect for the continuity correction now. –  xoedusk Jun 5 at 4:04

The actual chi-square statistic is discrete, not continuous - the chi-square distribution is an asymptotic continuous approximation.

If you simulate the distribution of the chi-square statistic under the null hypothesis, you can properly compare p-values.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.