Questions tagged [mcnemar-test]

A test for dependent categorical data, such as yes/no data before & after a treatment. The McNemar-Bowker test generalizes McNemar's test for $k>2$ categories. The tests assess whether the population proportions in each pair of corresponding symmetric cells are equal.

Filter by
Sorted by
Tagged with
29
votes
4answers
41k views

What is the difference between McNemar's test and the chi-squared test, and how do you know when to use each?

I have tried reading up on different sources, but I am still not clear what test would be the appropriate in my case. There are three different questions I am asking about my dataset: The subjects ...
14
votes
2answers
2k views

Relationship between McNemar's test and conditional logistic regression

I am interested in the modeling of binary response data in paired observations. We aim to make inference about the effectiveness of a pre-post intervention in a group, potentially adjusting for ...
9
votes
4answers
4k views

Fisher exact test on paired data

Given $40$ cases with lung cancer and $40$ matched controls (without lung cancer) (matching based on age, sex, etc.). To try to find evidence between the effect of smoking on lung cancer, I used ...
7
votes
1answer
817 views

McNemar or Fisher exact test for propensity score matched data?

I want to analyze some propensity score matched data. In the literature McNemar test is usually used, since the data is "paired". However matching is not pairing in the common sense. Would it be ...
7
votes
2answers
5k views

Importance of McNemar test in caret::confusionMatrix

There are many metrics to evaluate the performance of predictive model. Many of these appear relatively straightforward to me (e.g. Accuracy, Kappa, AUC-ROC, etc.) but I am uncertain regarding the ...
5
votes
3answers
592 views

Why aren't the diagonal counts used in McNemar's test?

In reviewing McNemar's test, I see that the diagonal counts aren't used in computing the test statistic. However, since the test is testing the null that the row and column marginals are equal, ...
5
votes
2answers
467 views

Can McNemar's test be improved upon by adjustments for zeros like those in a sign test?

McNemar's test is a special example of the binomial sign test, but the ('vanilla') sign test suffers from bias due to ignoring differences equal to zero. McNemar's test statistic is given by: $\chi^{...
5
votes
2answers
431 views

Can Cramér's V be used an effect size measure for McNemar's test?

I'm using McNemar's exact test (with R's "mcnemar.exact" function). I know that giving the odds ratio or proportion would be an adequate measure of effect size (Effect size of McNemar's Test), but ...
5
votes
2answers
755 views

Compare a diagnostic test to gold standard

I have a sample of patients who underwent 2 diagnostic tests, one of which is the gold standard. I've been asked to test if the diagnosis from the experimental test is different than the one from the ...
4
votes
1answer
851 views

Proper analyses for 2x2 contingency tables

So I have multiple questions about the different types of analyses that can be used on a 2x2 contingency table and when different analyses should be used. To open, I'll note that this stems from my ...
4
votes
2answers
911 views

Is the binomial sign test really the exact McNemar test?

I am calculating two different $p$ values for the McNemar test and what I believe is the corresponding exact test, even for combined counts larger than 25. Take, for example the contingency table <...
4
votes
1answer
11k views

McNemar-Bowker test for comparison of two measures

I am comparing two versions of a survey (an original, and a modified) and was wondering if the McNemar-Bowker test is appropriate. The survey(s) ask respondents to indicate whether they've had an ...
4
votes
1answer
488 views

Contingency table for gene groups

I have two lists of genes as follows: DEList has 282 gene names AllList has 32805 gene names ...
3
votes
1answer
1k views

McNemar's test vs chi-squared test for before & after data with different people from the same population

I have 2 proportions I want to compare. They are from the same population but at different time periods, in between which there was an intervention. H0: Proportion in Period#1 = Proportion in Period#...
3
votes
1answer
660 views

Why Yates correction in McNemar's Test subtract 1?

McNemar's test statistic is given by: Yates's correction defines something like this: However, the correction for the McNemar's test statistic requires 1 (not 0.5)...
3
votes
1answer
90 views

Which test should I use to assess for the statistical signficance of changes in multiple binomial dependent variables from T1 to T2?

Students have completed a test containing 20 questions at both T1 and T2, with an intervention in the interval. Scores for each question are either 0 (incorrect) or 1 (correct). I am interested in ...
3
votes
1answer
45 views

Can I perform a McNemar's test with only proportion and denominator information?

I have the following count and denominator information. The 630 subjects are the same people over two different periods of time. Can I conduct a McNemar's test to compare the dependent proportion ...
3
votes
2answers
146 views

McNemar's test probably does not work, but what could?

I have a dataset of people who had back-to-back procedures and I have the whether or not polyps (precancerous lesions) removed during each procedure. ...
3
votes
1answer
126 views

McNemar's Test for repeated sex ratio data

I have proportion data (sex ratio of a dioecious plant) for a population that was measured three times. I would like to know if a McNemar's Test is the most appropriate analysis to answer the question:...
3
votes
1answer
2k views

Sample size function for McNemar test of repeated proportions in R

I have been scouring CRAN and Google for a function (that I wouldn't have to code myself) that would calculate the sample size needed for a pre-post design with a dichotomous outcome - to no avail. If ...
3
votes
0answers
37 views

Does sampling fraction matter for McNemar's Test?

For McNemar's Test, does the sample need to be less than 1/20 of the population if the sample is without replacement? If so, can I correct for a sample that is larger than 1/20 of the population by ...
3
votes
0answers
1k views

Chi-square test vs. McNemar's test - which test is right for this highly usual situation?

Friends, Here is, I would assume, a fairly common situation: I would like to know which test to use to assess whether the tendency to answer "Yes" on question 1 of a questionnaire is associated with ...
3
votes
0answers
429 views

Pearson's chi-squared test assumption: independence of observations

I am unsure of how to establish (or interpret) the independence condition of the chi-squared test in (for instance) the following situation. Consider this 2x2 contingency table representing the ...
3
votes
0answers
121 views

Does conditional logistic regression effectively “throw away” concordant pairs like McNemar's test?

We are interested in analyzing pre-post data from a randomized control trial. The outcome is binary and I am trying to explain the heuristics of the model formulation for conditional logistic ...
2
votes
2answers
4k views

Determine if two proportions from the same sample are different

Is there a test I can use to compare two proportions from the same sample that are not mutually exclusive? For example, I would like see if the proportion of people that answered "yes" to a ...
2
votes
1answer
781 views

One-sided McNemar's test

In R, the function mcnemar.test has the following example: ...
2
votes
1answer
1k views

How to calculate information included in R's confusion matrix

I want know the formulas for the information highlighted in the confusion matrix in the picture below:
2
votes
2answers
817 views

McNemar's test p-value NA

I have paired data from 9 individuals. The response variable is ordinal with levels 1-5. As the sample is very small and the data are categorical, I thought the appropriate test was McNemar's test. ...
2
votes
1answer
236 views

McNemar test comparing two proportions in a crossover trial

In a crossover trial involving 50 patients, two dialysis treatment schedules were compared.Ten patients complained of vomiting on both schedules, 30 on neither, nine on treatment A only and one on ...
2
votes
1answer
4k views

Paired comparison chi square

I have a question related to the use of chi square test for paired data. I have read that McNemar tests would be an option but in some software like R it works only for 2x2 data (may be I do not know ...
2
votes
3answers
91 views

Hypothesis test for intervention with class imbalance before intervention. Alternative to McNemar

I want to assess an intervention to cure a disease. An example is described by the following matrix: ...
2
votes
1answer
584 views

comparing proportions before and after two different interventions

I am trying to compare the proportions before and after TWO interventions in a small sample. Basically, 10 people had intervention A, and I want to compare proportions of their success rate before ...
2
votes
1answer
623 views

Comparing two decision models using McNemar's test

I'm trying to compare the performance of two classification algorithms. Thanks to CrossValidated, I already know that the best way to do this is via McNemar's test. I performed this, and sure enough, ...
2
votes
1answer
205 views

What is the Bayesian counterpart to the McNemar test (of marginal homogeneity)?

In analogy to a similar post on continuous data, I ask the following. In paired discrete data, we test the frequentist hypothesis of equivalent marginal distributions of two discrete variables $X$ and ...
2
votes
1answer
130 views

Test for pairs of small integers

I have two classifiers, say $A$ and $B$ and I am performing 10$\times$10-fold cross-validation on them and record the number of instances correctly classified from the test set. Unfortunately, the ...
2
votes
1answer
3k views

Cochran's Q & McNemar tests together

I have binomial outcome data for three groups that I am analyzing with Cochran's Q test. As discussed at this site, Cochran. However, I have to repeat this seven different times because I am ...
2
votes
1answer
114 views

Determine if an action increases the proportion of 1s in binary data with unknown population

We're trying to improve search results, and we're trying to determine if certain changes (adding a word to a synonym list, removing it from the query, etc) have a statistically significant improvement ...
2
votes
1answer
1k views

McNemar's test for $3 \times 2$ data

I adopt a within-group experiment, such that the same participant uses 3 devices to perform a task, which has a result "pass" or fail. My data looks like this: ...
2
votes
1answer
637 views

Is McNemar's Test appropriate for evaluating pre-post differences in intervention studies with a control arm?

I recently came across a paper that uses the McNemar test to evaluate the effectiveness of an intervention to improve adherence to treatment. The study used a pre/post test design whereby the ...
2
votes
1answer
503 views

Analyzing multiple dependent pairs of 2x2 contingency tables

In advance, I ask forgiveness for my statistical naïvety. Suppose I have experimental data from pre- and post-treatment, and this data is split according to two binary categories like so: \begin{...
2
votes
2answers
500 views

How to get random classification to assess the performance of classifier with McNemar test?

I'm trying to replicate a study where the author used the McNemar test to assess the performance of classification compared to random classification. I have the original classifier and I'm using R to ...
2
votes
1answer
195 views

McNemars Test Sample size calculation

I've searched for this particular question but i can't seem to find the right answer. Lets say i expect prob p01 = 0.2 and prob p10 = 0.3. I want a power of 0.8 and alpha of 0.05. This website: ...
2
votes
1answer
189 views

Which test to use? Repeated measures with binary outcomes

I would like your opinion about which analyses to use for the research question. I've got a repeated measures with binary data. N = 20 H0 = there is no difference between pre and post measurement ...
2
votes
0answers
366 views

Can Cochran's Q test take a 3 level nominal variable as the response (instead of 2)?

I conducted a study using within-subjects (repeated measure) design for collecting nominal data. I have been reading a lot closet I get for a suitable test is Cochran's Q test since I have repeated ...
2
votes
1answer
295 views

Is there a Bowker-like test to assess the symmetry of squared contingency tables with small expected values and a small number of observations?

I ask this question because Bowker test (McNemar extension for $k\times k$ tables with $k>2$) requires a large number of observations for fitting well and also requires (as a rule of thumb) that ...
2
votes
0answers
218 views

Comparison of multiple diagnostic tests with paired data

Good Day All, I am attempting to compare three diagnostic tests against a gold-standard and for simplicity sake, let us have all the outcomes be binary (disease/no disease). In this study study, all ...
2
votes
0answers
253 views

Comparing proportions from the (somewhat) same sample

I’m looking for some help on testing whether there are statistical differences between proportions when the data is dependent. By dependent, I mean that all the data is from the same participants ...
1
vote
2answers
50 views

What statistical test would I use to test that both data sources are matching [closed]

Suppose I am only working with two datasets, Data Source 1 with columns Customer ID and ...
1
vote
2answers
2k views

Which test to choose when Wilcoxon test and McNemar's test are different?

I have paired data.The response variable is categorical with levels 1-5. And I have used both McNemar's test and Wilcoxon rank sign test and I get different p-values. For McNemar's test a p-value of 0....
1
vote
1answer
93 views

Site interaction with McNemar tests [closed]

I have some clinical diagnostic data from three sites that I have been looking at performance against a reference standard using the McNemar test. How would I set up the analysis to investigate if ...