# Where does p-value come from in McNemar's Test?

I was using this online calculator to work out McNemar's Test on this data set:

I get the results:

The two-tailed P value equals 0.1859

Chi squared equals 1.750 with 1 degrees of freedom.

Now I understand how to calculate the Chi squared but I have no idea where this P value comes from.

Also how does the P value relate to whether there is a significant difference between classifiers?

If someone could explain this like a program in JavaScript or something it would be so much clearer. Every explanation I see throws jargon all over the place.

Asymptotically the McNemar test statistic follows a chi-squared distribution with 1 degree of freedom. So, if $x_{obs}$ is your observed McNemar test statistic, the $p$ value is

$p = \text{Pr}\left\{ \chi^2_1 > x_{obs}\right\}$

but perhaps this is all the jargon and what not that you were saying you were confused about. What the statement above is saying is that the $p$ value is a probability calcuated under the chi-squared $(\chi^2_1)$ distribution. You can think of a probability as being an area under a particular curve (think back to integrals in calculus).

The curve in question here is the chi-squared density

The $p$ value is the blue shaded area under the curve I have plotted.

The curve is defined by: $f(x)=\dfrac{e^{-x/2}}{\sqrt{2x}\Gamma(1/2)}$

Now sure you can try to calculate this by hand to get p-value but most programming languages have built in functions to calculate are under density curves. In R you can do this:

> pchisq(1.75, df=1,lower.tail=FALSE)
[1] 0.1858767

• Sorry, are you sure that it is a two-sided p-value? – Sergio May 17 '14 at 21:13
• The p value noted in OP's question looks like one-sided because it matches the one-sided calculation i did in R. – bdeonovic May 17 '14 at 21:24
• I'd say that it is one-sided, because the chi-squared support is $[0,+\infty]$, so pchisq(-1.75,1)=0. Am I wrong? – Sergio May 17 '14 at 21:29
• I'd fell better if $p=2\text{Pr}(\chi_1^2>1.75)=0.1859$ ;-) – Sergio May 17 '14 at 22:37
• Regardless of semantics (is the test two-sided because it has power against departures from the null hypothesis in either of two directions, or one-sided because one tail area of the test statistic's density gives its size?) the expression $p = 2\text{Pr}\left\{ \chi^2_1 > x_{obs}\right\}$ is simply wrong - out by a factor of 2 as @Sergio says. – Scortchi May 18 '14 at 0:29