46

It is very unfortunate that McNemar's test is so difficult for people to understand. I even notice that at the top of its Wikipedia page it states that the explanation on the page is difficult for people to understand. The typical short explanation for McNemar's test is either that it is: 'a within-subjects chi-squared test', or that it is 'a test of the ...


22

Well, it seems I've made a hash of this. Let me try to explain this again, in a different way and we'll see if it might help clear things up. The traditional way to explain McNemar's test vs. the chi-squared test is to ask if the data are "paired" and to recommend McNemar's test if the data are paired and the chi-squared test if the data are "unpaired". ...


11

The question of which test to use, contingency table $\chi^{2}$ versus McNemar's $\chi^{2}$ of a null hypothesis of no association between two binary variables is simply a question of whether your data are paired/dependent, or unpaired/independent: Binary Data in Two Independent Samples In this case, you would use a contingency table $\chi^{2}$ test. For ...


8

You need McNemar's test (http://en.wikipedia.org/wiki/McNemar%27s_test , http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3346204/). Following is an example: 1300 pts and 1300 matched controls are studied. The smoking status is tabled as follows: Normal |no |yes| Cancer|No |1000|40 | |Yes |200 |60 | Each entry of the table ...


8

This is definitely an ongoing debate in the literature, but at this point the evidence points to using paired analysis to compute standard errors and p-values. Although the goal of matching is to arrive at two samples that mimic a randomized control trial, not a paired-randomized control trial, matching does still induce a covariance between the outcomes ...


7

My understanding of McNemar's test is as follows: It is used to see whether an intervention has made a significant difference to a binary outcome. In your example, a group of subjects are checked for infection and the response is recorded as yes or no. All subjects are then given some intervention, say an antibiotic drug. They are then checked again for ...


6

E.L. Lehmann, J.P. Romano. Testing Statistical Hypotheses. 3rd ed. Springer, 2005. P. 136: P(-), P(+) and P(0) denote the probabilities of preference for product [A over B, B over A, or A=B, tie], ... The hypothesis to be tested H0: P(+)=P(-) ... The problem reduces to that of testing the hypothesis P=1/2 in a binomial distribution with n-z [n = ...


5

Pearson's $\chi^2$ test is useful for a sample of $n$ observations cross-classified by two variables, say $A$ and $B$. These tests test the null hypothesis that $A$ and $B$ are independent variables. So, for an example, if you crossed two strains of D. melanogaster (fruit flies) with different mutations and observed the $F_2$ generation frequencies in $n$ ...


5

You are right that Fisher's exact test is inappropriate for your data. You will have to re-form your contingency table. The new table will be for pairs, thus it will appear to have half as many data represented (in your case 40 instead of 80). For example, imagine your data looked like this (each set of paired subjects is in its own row, and 1 indicates a ...


5

The within-subjects test of equality of proportions is McNemar's test. I discuss it fairly thoroughly here: What is the difference between McNemar's test and the chi-squared test, and how do you know when to use each? The answer also provides an example demo in R, should you use that, but it should be easy to do in any software.


5

Asymptotically the McNemar test statistic follows a chi-squared distribution with 1 degree of freedom. So, if $x_{obs}$ is your observed McNemar test statistic, the $p$ value is $p = \text{Pr}\left\{ \chi^2_1 > x_{obs}\right\}$ but perhaps this is all the jargon and what not that you were saying you were confused about. What the statement above is ...


5

The tag info for McNemar's test (once supplied by me, and later possibly modified): A repeated-measures test for categorical data. Given that two variables with the same 2 categories (McNemar test) or k categories (McNemar-Bowker test) form a square contingency table, the test's question is whether population proportion in every off-diagonal cell ...


4

Two points: 1. Continuity Correction McNemar's test uses discrete data (counts of discordant pairs) to produce a $\chi^{2}$ test statistic with one degree of freedom. However, the $\chi^{2}$ distribution is not discrete, but continuous. Hence, the need for a continuity correction. $\chi^{2}=\frac{\left(\left|50-65\right|-1\right)^{2}}{\left(50+65\right)} =...


4

Sorry, it's an old issue, I came across this by chance. There is a mistake in your code for the mcnemar test. Try with: n <- 100 do.one <- function(n) { id <- rep(1:n, each=2) case <- rep(0:1, times=n) rs <- rbinom(n*2, 1, 0.5) c( 'pclogit' = coef(summary(clogit(case ~ rs + strata(id))))[5], 'pmctest' = mcnemar.test(table(rs[...


4

I think you're looking at this the wrong way. You're trying to compare the proportion of insects left after applying insecticide. The 'before' aren't a random sample, but the experimental setup. That is: \begin{array}{l|c|c|c} & &\text{count left }&\\ &n \text{ exposed} &\text{after insecticide}&\text{proportion left}\\ \hline \text{...


4

Intuitively, since "Cell $a$" is in the first row and first column, including $a$ in your test wouldn't help to assess the difference between the row marginal and the column marginal. If your row is $a+b$ and your column is $a+c$, to test if $a+b=a+c$, you only need to test $b=c$. Including $a$ is irrelevant to the extent that we want to test for equality ...


4

The way to check is to create the contingency table: \begin{array}{rccl} &\text{treatment B} &\text{vomiting} & \\ \text{treatment A vomiting }\ \ &{\rm yes} &{\rm no} &{\rm sum} \\ {\rm yes} &10 &9 &? \\ {\rm no} &1 &30 &31 \\ {\rm sum} &? &39 & \end{array} An overview of McNemar's test, it'...


4

library(caret) mat = matrix(c(55,34,56,255), ncol=2, byrow=TRUE) mat # output omitted confusionMatrix(as.table(mat), positive="B") # Confusion Matrix and Statistics # # A B # A 55 34 # B 56 255 # # Accuracy : 0.775 # 95% CI : (0.7309, 0.815) # No Information ...


4

I have described the gist of McNemar's test rather extensively here and here, it may help you to read those. Briefly, McNemar's test assesses the balance of the off-diagonal counts. If people were as likely to transition from approval to disapproval as from disapproval to approval, then the off-diagonal values should be approximately the same. The ...


4

Cramér's $V$ doesn't correspond well to what is tested by McNemar's test. Edit: Disclosure: the webpage and R package cited below are mine. Probably the most common effect size statistic for McNemar's test is odds ratio, though Cohen's $g$ could be used. Cohen (1988) also uses a statistic he calls $P$. For defintions, quoting from here: Considering a ...


3

You can use multi-level logistic regression. You've only got one dependent variable, correctness. You have multiple independent variables nested within student. In R you can use lmer to construct the model. It would look something like. m <- lmer( answer ~ treatment * Q + (treatment * Q | student), family = 'binomial', data = mydata) That would allow ...


3

McNemar-Bowker test of symmetry of k X k contingency table is inherently 2-sided: the alternative hypothesis is undirected. So, in general case it cannot be used to test a one sided alternative that subdiagonal frequencies are larger/smaller than superdiagonal frequencies. But since in your case the differences are consistently in favour of subdiagonal ...


3

The uncorrected McNemar statistic is the square of a standardized* difference in two counts; the continuity correction is $\frac12$ to both, but it's $-\frac12$ for the larger of the two and $+\frac12$ for the smaller of the two[1]. * under the null As a result, when you take $|B-C|$ the total of the two continuity corrections is always $-1$. (This post ...


3

Since you don't have the same people in both samples, you should use the chi-squared test. You would set it up like this: Period 1 Period 2 yes n_y1 n_y2 no n_n1 n_n2 where n_rp is the number of people with the given response in the specified time period. At this point, I usually tell people to use the $z$-test for the ...


3

I wonder if you really want McNemar's test (more specifically the McNemar-Bowker test). These are tests to see if the marginal proportions are the same (see here). Under the assumption that pairings can be recovered from the orders of the two vectors, here is a table of your data with the marginal proportions computed: bef = c(4, 3, 4, 5, 4, 4, 4, 5,...


3

McNemar's test is for paired data but CMH is not. What kinds of differences are you looking for? If you're interested in heterogeneity of the prevalence of the outcome, fit a logistic model with site effects for the reference values only (this will give you independent data). If you're interested in heterogeneity in the agreements, just mutually compare the ...


3

First, a p-value of 0.83 means you do not reject the null. Second, McNemar's test is about whether the row and column marginals are equal, or, equivalently, whether the "off-diagonal" elements are equal. Since your p value is quite high, you cannot reject the null that they are equal. It's not clear, from your question, what was in the four cells of the ...


3

If you use McNemar's test you are testing whether the table is symmetric: whether more people are diagnosed sick by the new method and well by the old versus well by the new and sick by the old. This is a perfectly reasonable scientific question to have.For a concrete situation suppose the two methods being compared are ratings of mental health problems by a ...


3

In the context of a 2x2 table, Cramer's $V$ is equivalent to the phi coefficient. Moreover, phi is equivalent to Pearson's product moment correlation of the two columns of $1$'s and $0$'s when the 2x2 table is disaggregated. That corresponds to a different magnitude than the one that McNemar's test is testing. So, no, I don't think it is a good choice. ...


3

If you find a single mismatched pair, you know the data sets are not matched. It's impossible both to find a mismatched pair and for the datasets to be perfectly matched. So any test that relies on finding any mismatched pairs will have a false alarm rate (i.e., alpha) of 0. The power of any test depends on the true "effect". The datasets would be ...


Only top voted, non community-wiki answers of a minimum length are eligible