Questions tagged [mcnemar-test]

A test for dependent categorical data, such as yes/no data before & after a treatment. The McNemar-Bowker test generalizes McNemar's test for $k>2$ categories. The tests assess whether the population proportions in each pair of corresponding symmetric cells are equal.

Filter by
Sorted by
Tagged with
30
votes
4answers
46k views

What is the difference between McNemar's test and the chi-squared test, and how do you know when to use each?

I have tried reading up on different sources, but I am still not clear what test would be the appropriate in my case. There are three different questions I am asking about my dataset: The subjects ...
14
votes
2answers
3k views

Relationship between McNemar's test and conditional logistic regression

I am interested in the modeling of binary response data in paired observations. We aim to make inference about the effectiveness of a pre-post intervention in a group, potentially adjusting for ...
9
votes
4answers
4k views

Fisher exact test on paired data

Given $40$ cases with lung cancer and $40$ matched controls (without lung cancer) (matching based on age, sex, etc.). To try to find evidence between the effect of smoking on lung cancer, I used ...
8
votes
2answers
7k views

Importance of McNemar test in caret::confusionMatrix

There are many metrics to evaluate the performance of predictive model. Many of these appear relatively straightforward to me (e.g. Accuracy, Kappa, AUC-ROC, etc.) but I am uncertain regarding the ...
7
votes
1answer
1k views

McNemar or Fisher exact test for propensity score matched data?

I want to analyze some propensity score matched data. In the literature McNemar test is usually used, since the data is "paired". However matching is not pairing in the common sense. Would it be ...
6
votes
3answers
709 views

Why aren't the diagonal counts used in McNemar's test?

In reviewing McNemar's test, I see that the diagonal counts aren't used in computing the test statistic. However, since the test is testing the null that the row and column marginals are equal, ...
6
votes
2answers
819 views

Can Cramér's V be used an effect size measure for McNemar's test?

I'm using McNemar's exact test (with R's "mcnemar.exact" function). I know that giving the odds ratio or proportion would be an adequate measure of effect size (Effect size of McNemar's Test), but ...
5
votes
2answers
535 views

Can McNemar's test be improved upon by adjustments for zeros like those in a sign test?

McNemar's test is a special example of the binomial sign test, but the ('vanilla') sign test suffers from bias due to ignoring differences equal to zero. McNemar's test statistic is given by: $\chi^{...
5
votes
2answers
2k views

Compare a diagnostic test to gold standard

I have a sample of patients who underwent 2 diagnostic tests, one of which is the gold standard. I've been asked to test if the diagnosis from the experimental test is different than the one from the ...
5
votes
1answer
12k views

McNemar-Bowker test for comparison of two measures

I am comparing two versions of a survey (an original, and a modified) and was wondering if the McNemar-Bowker test is appropriate. The survey(s) ask respondents to indicate whether they've had an ...
4
votes
1answer
954 views

Proper analyses for 2x2 contingency tables

So I have multiple questions about the different types of analyses that can be used on a 2x2 contingency table and when different analyses should be used. To open, I'll note that this stems from my ...
4
votes
2answers
1k views

Is the binomial sign test really the exact McNemar test?

I am calculating two different $p$ values for the McNemar test and what I believe is the corresponding exact test, even for combined counts larger than 25. Take, for example the contingency table <...
4
votes
1answer
66 views

Can I perform a McNemar's test with only proportion and denominator information?

I have the following count and denominator information. The 630 subjects are the same people over two different periods of time. Can I conduct a McNemar's test to compare the dependent proportion ...
4
votes
1answer
562 views

Contingency table for gene groups

I have two lists of genes as follows: DEList has 282 gene names AllList has 32805 gene names ...
3
votes
2answers
5k views

Determine if two proportions from the same sample are different

Is there a test I can use to compare two proportions from the same sample that are not mutually exclusive? For example, I would like see if the proportion of people that answered "yes" to a ...
3
votes
1answer
1k views

McNemar's test vs chi-squared test for before & after data with different people from the same population

I have 2 proportions I want to compare. They are from the same population but at different time periods, in between which there was an intervention. H0: Proportion in Period#1 = Proportion in Period#...
3
votes
1answer
790 views

Why Yates correction in McNemar's Test subtract 1?

McNemar's test statistic is given by: Yates's correction defines something like this: However, the correction for the McNemar's test statistic requires 1 (not 0.5)...
3
votes
1answer
91 views

Which test should I use to assess for the statistical signficance of changes in multiple binomial dependent variables from T1 to T2?

Students have completed a test containing 20 questions at both T1 and T2, with an intervention in the interval. Scores for each question are either 0 (incorrect) or 1 (correct). I am interested in ...
3
votes
1answer
44 views

Is McNemar's Test Appropriate Here?

I am applying two diagnostic tests with dichotomous outcomes (condition present yes/no) to a sample of patient data. Both take into account a range of vital parameters and produce a test result, which ...
3
votes
2answers
177 views

McNemar's test probably does not work, but what could?

I have a dataset of people who had back-to-back procedures and I have the whether or not polyps (precancerous lesions) removed during each procedure. ...
3
votes
1answer
137 views

McNemar's Test for repeated sex ratio data

I have proportion data (sex ratio of a dioecious plant) for a population that was measured three times. I would like to know if a McNemar's Test is the most appropriate analysis to answer the question:...
3
votes
1answer
2k views

Sample size function for McNemar test of repeated proportions in R

I have been scouring CRAN and Google for a function (that I wouldn't have to code myself) that would calculate the sample size needed for a pre-post design with a dichotomous outcome - to no avail. If ...
3
votes
0answers
37 views

Does sampling fraction matter for McNemar's Test?

For McNemar's Test, does the sample need to be less than 1/20 of the population if the sample is without replacement? If so, can I correct for a sample that is larger than 1/20 of the population by ...
3
votes
0answers
1k views

Chi-square test vs. McNemar's test - which test is right for this highly usual situation?

Friends, Here is, I would assume, a fairly common situation: I would like to know which test to use to assess whether the tendency to answer "Yes" on question 1 of a questionnaire is associated with ...
3
votes
0answers
474 views

Pearson's chi-squared test assumption: independence of observations

I am unsure of how to establish (or interpret) the independence condition of the chi-squared test in (for instance) the following situation. Consider this 2x2 contingency table representing the ...
3
votes
0answers
148 views

Does conditional logistic regression effectively “throw away” concordant pairs like McNemar's test?

We are interested in analyzing pre-post data from a randomized control trial. The outcome is binary and I am trying to explain the heuristics of the model formulation for conditional logistic ...
2
votes
1answer
2k views

How to calculate information included in R's confusion matrix

I want know the formulas for the information highlighted in the confusion matrix in the picture below:
2
votes
1answer
1k views

One-sided McNemar's test

In R, the function mcnemar.test has the following example: ...
2
votes
2answers
1k views

McNemar's test p-value NA

I have paired data from 9 individuals. The response variable is ordinal with levels 1-5. As the sample is very small and the data are categorical, I thought the appropriate test was McNemar's test. ...
2
votes
1answer
305 views

McNemar test comparing two proportions in a crossover trial

In a crossover trial involving 50 patients, two dialysis treatment schedules were compared.Ten patients complained of vomiting on both schedules, 30 on neither, nine on treatment A only and one on ...
2
votes
1answer
4k views

Paired comparison chi square

I have a question related to the use of chi square test for paired data. I have read that McNemar tests would be an option but in some software like R it works only for 2x2 data (may be I do not know ...
2
votes
3answers
160 views

Hypothesis test for intervention with class imbalance before intervention. Alternative to McNemar

I want to assess an intervention to cure a disease. An example is described by the following matrix: ...
2
votes
1answer
395 views

Compare classification performance of two heuristics

I want to compare the classification performance of two heuristics: $h_0$ and $h_1$. I have around 700 samples for which I know the output class $c$ and the results of the two heuristics $c_{\textit{...
2
votes
1answer
4k views

Fisher's exact test in 2x3 contingency table

I have a dataset with birds that were exposed to pesticides. The summary of this data looks like: ...
2
votes
1answer
749 views

comparing proportions before and after two different interventions

I am trying to compare the proportions before and after TWO interventions in a small sample. Basically, 10 people had intervention A, and I want to compare proportions of their success rate before ...
2
votes
1answer
917 views

Comparing two decision models using McNemar's test

I'm trying to compare the performance of two classification algorithms. Thanks to CrossValidated, I already know that the best way to do this is via McNemar's test. I performed this, and sure enough, ...
2
votes
1answer
243 views

What is the Bayesian counterpart to the McNemar test (of marginal homogeneity)?

In analogy to a similar post on continuous data, I ask the following. In paired discrete data, we test the frequentist hypothesis of equivalent marginal distributions of two discrete variables $X$ and ...
2
votes
1answer
292 views

Test if treatment effect is different between samples

I want to determine if there is significant variation in the effect of a treatment amongst my replicates. The treatment effect is measured as a proportion. Let me describe: I study heart defects (in ...
2
votes
1answer
140 views

Test for pairs of small integers

I have two classifiers, say $A$ and $B$ and I am performing 10$\times$10-fold cross-validation on them and record the number of instances correctly classified from the test set. Unfortunately, the ...
2
votes
1answer
3k views

Cochran's Q & McNemar tests together

I have binomial outcome data for three groups that I am analyzing with Cochran's Q test. As discussed at this site, Cochran. However, I have to repeat this seven different times because I am ...
2
votes
1answer
117 views

Determine if an action increases the proportion of 1s in binary data with unknown population

We're trying to improve search results, and we're trying to determine if certain changes (adding a word to a synonym list, removing it from the query, etc) have a statistically significant improvement ...
2
votes
1answer
1k views

McNemar's test for $3 \times 2$ data

I adopt a within-group experiment, such that the same participant uses 3 devices to perform a task, which has a result "pass" or fail. My data looks like this: ...
2
votes
1answer
732 views

Is McNemar's Test appropriate for evaluating pre-post differences in intervention studies with a control arm?

I recently came across a paper that uses the McNemar test to evaluate the effectiveness of an intervention to improve adherence to treatment. The study used a pre/post test design whereby the ...
2
votes
1answer
82 views

Chi Square test or McNemar test?

I've got a set of 1000 patients and I am measuring two different blood values (CRP and IL6) and choosing a categorical cutoff values for both (CRP>10 and IL6 >2). I now would like to know if there is ...
2
votes
1answer
1k views

McNemar test with multiple scores for the same subject

I am having a statistical issue: subjects receive a test with machine A and machine B. 3 experts classify whether the results of machine A and machine B are acceptable (1) or not (0). From my ...
2
votes
1answer
558 views

Analyzing multiple dependent pairs of 2x2 contingency tables

In advance, I ask forgiveness for my statistical naïvety. Suppose I have experimental data from pre- and post-treatment, and this data is split according to two binary categories like so: \begin{...
2
votes
1answer
329 views

Is there a Bowker-like test to assess the symmetry of squared contingency tables with small expected values and a small number of observations?

I ask this question because Bowker test (McNemar extension for $k\times k$ tables with $k>2$) requires a large number of observations for fitting well and also requires (as a rule of thumb) that ...
2
votes
2answers
556 views

How to get random classification to assess the performance of classifier with McNemar test?

I'm trying to replicate a study where the author used the McNemar test to assess the performance of classification compared to random classification. I have the original classifier and I'm using R to ...
2
votes
1answer
419 views

McNemars Test Sample size calculation

I've searched for this particular question but i can't seem to find the right answer. Lets say i expect prob p01 = 0.2 and prob p10 = 0.3. I want a power of 0.8 and alpha of 0.05. This website: ...
2
votes
1answer
252 views

Which test to use? Repeated measures with binary outcomes

I would like your opinion about which analyses to use for the research question. I've got a repeated measures with binary data. N = 20 H0 = there is no difference between pre and post measurement ...