8 votes
Accepted

McNemar or Fisher exact test for propensity score matched data?

This is definitely an ongoing debate in the literature, but at this point the evidence points to using paired analysis to compute standard errors and p-values. Although the goal of matching is to ...
Noah's user avatar
  • 33.4k
7 votes
Accepted

How to calculate information included in R's confusion matrix

...
gung - Reinstate Monica's user avatar
7 votes

Can Cramér's V be used an effect size measure for McNemar's test?

Cramér's $V$ doesn't correspond well to what is tested by McNemar's test. Edit: Disclosure: the webpage and R package cited below are mine. Probably the most common effect size statistic for McNemar'...
Sal Mangiafico's user avatar
6 votes
Accepted

One-sided McNemar's test

I have described the gist of McNemar's test rather extensively here and here, it may help you to read those. Briefly, McNemar's test assesses the balance of the off-diagonal counts. If people were ...
gung - Reinstate Monica's user avatar
5 votes
Accepted

Why aren't the diagonal counts used in McNemar's test?

The tag info for McNemar's test (once supplied by me, and later possibly modified): A repeated-measures test for categorical data. Given that two variables with the same 2 categories (McNemar ...
ttnphns's user avatar
  • 57.6k
5 votes
Accepted

Replacing McNemar test with regression?

The following example will work to model with logistic regression a test similar to McNemar's test on a 2 x 2 table. Essentially the dependent variable for a binomial logistic model takes on one value ...
Sal Mangiafico's user avatar
4 votes

Importance of McNemar test in caret::confusionMatrix

Interpret the McNemar’s Test for Classifiers McNemar’s Test captures the errors made by both models. Specifically, the No/Yes and Yes/No (A/B and B/A in your case) cells in the confusion matrix. The ...
Aman Mathur's user avatar
4 votes

Why aren't the diagonal counts used in McNemar's test?

Intuitively, since "Cell $a$" is in the first row and first column, including $a$ in your test wouldn't help to assess the difference between the row marginal and the column marginal. If your row is $...
Matt Brems's user avatar
  • 2,863
4 votes
Accepted

McNemar test comparing two proportions in a crossover trial

The way to check is to create the contingency table: \begin{array}{rccl} &\text{treatment B} &\text{vomiting} & \\ \text{treatment A vomiting }\ \ &{\rm yes} &{\rm no} &{\rm ...
gung - Reinstate Monica's user avatar
4 votes
Accepted

Relationship between statistical models: mcnemar test vs logistic regression

Because your dependent variable is ordinal in nature, ordinal regression is probably your best bet. Luckily the ordinal package in R allows for fixed effects and mixed effects models. If I ...
Sal Mangiafico's user avatar
4 votes

What statistical test would I use to test that both data sources are matching

If you find a single mismatched pair, you know the data sets are not matched. It's impossible both to find a mismatched pair and for the datasets to be perfectly matched. So any test that relies on ...
Noah's user avatar
  • 33.4k
4 votes
Accepted

If we use McNemar's for single case data, is pseudoreplication an issue?

Yes, you can use your lower matrix in McNemar's test to assess if the intervention (or anything that is associated with it, see below) changed the likelihood of person A to correctly respond to items. ...
stefgehrig's user avatar
  • 1,149
3 votes
Accepted

Comparing two decision models using McNemar's test

Slide 22 is a bit confusing. The error rates on that slide are usually better described as positive and negative test results. Before talking about false positives (FP), true negatives (TN), etc. let ...
Carl's user avatar
  • 13.1k
3 votes
Accepted

McNemar's test vs chi-squared test for before & after data with different people from the same population

Since you don't have the same people in both samples, you should use the chi-squared test. You would set it up like this: ...
gung - Reinstate Monica's user avatar
3 votes
Accepted

McNemar test with multiple scores for the same subject

You want Cochran's Q test. Just as the one way repeated measures ANOVA generalizes the paired t test to two or more measurements in the same individual (or more than two individuals matched per block),...
Alexis's user avatar
  • 29.9k
3 votes

Site interaction with McNemar tests

McNemar's test is for paired data but CMH is not. What kinds of differences are you looking for? If you're interested in heterogeneity of the prevalence of the outcome, fit a logistic model with site ...
AdamO's user avatar
  • 62.7k
3 votes
Accepted

McNemar's test p-value NA

I wonder if you really want McNemar's test (more specifically the McNemar-Bowker test). These are tests to see if the marginal proportions are the same (see here). Under the assumption that pairings ...
gung - Reinstate Monica's user avatar
3 votes
Accepted

Why Yates correction in McNemar's Test subtract 1?

The uncorrected McNemar statistic is the square of a standardized* difference in two counts; the continuity correction is $\frac12$ to both, but it's $-\frac12$ for the larger of the two and $+\...
Glen_b's user avatar
  • 283k
3 votes

Importance of McNemar test in caret::confusionMatrix

This is an interesting question that has different answers according to the context. I agree with what have answered you before, so here I will focus more on the context. It is normal that you have ...
Pavel Novoa's user avatar
3 votes
Accepted

Mcnemar's test p-value output in R confusion matrix

First, a p-value of 0.83 means you do not reject the null. Second, McNemar's test is about whether the row and column marginals are equal, or, equivalently, whether the "off-diagonal" elements are ...
Peter Flom's user avatar
  • 120k
3 votes
Accepted

Compare a diagnostic test to gold standard

If you use McNemar's test you are testing whether the table is symmetric: whether more people are diagnosed sick by the new method and well by the old versus well by the new and sick by the old. This ...
mdewey's user avatar
  • 17.8k
3 votes

Compare a diagnostic test to gold standard

You are asking about agreement, so you should use a test for agreement. With just two diagnostic measures ('raters') that are categorical in nature, the standard test is Cohen's kappa. Here's a ...
gung - Reinstate Monica's user avatar
3 votes
Accepted

Can Cramér's V be used an effect size measure for McNemar's test?

In the context of a 2x2 table, Cramer's $V$ is equivalent to the phi coefficient. Moreover, phi is equivalent to Pearson's product moment correlation of the two columns of $1$'s and $0$'s when the ...
gung - Reinstate Monica's user avatar
3 votes
Accepted

Is McNemar's Test Appropriate Here?

You could use McNemar's test. You do (potentially) have a 2x2 table. For each patient, you have the output from the two diagnostic tests, and the knowledge of the 'correct answer' from the gold ...
gung - Reinstate Monica's user avatar
3 votes
Accepted

Interpretation of McNemar test example

Your data should be something like this: ...
StupidWolf's user avatar
  • 5,107
3 votes
Accepted

How to derive sampling variance of paired differences?

To derive the sampling distribution of any statistics, you need to specify the underlying distribution first. For the paired data, although the paper you linked didn't make this clear, the joint ...
Zhanxiong's user avatar
  • 18.9k
2 votes
Accepted

Test for pairs of small integers

This problem calls for using a permutation test, a pure non-parametric test that does not have any normality assumptions. Check the "Hot Wings" example in Section 16 of these notes: http://arxiv.org/...
Kostia 's user avatar
  • 1,607
2 votes

Why aren't the diagonal counts used in McNemar's test?

Your intuition is reasonable, but incorrect. (As you note, that state of affairs is very common in probability and statistics.) Statistical significance vs. effect size / practical significance You ...
gung - Reinstate Monica's user avatar
2 votes

McNemar's test p-value NA

It is almost certainly because you have a pair of cells which both have zeroes (3, 5) and (5, 3). If you look at the way the test is defined that will lead to problems. What to do is a problem. You ...
mdewey's user avatar
  • 17.8k
2 votes

Which test to choose when Wilcoxon test and McNemar's test are different?

McNemar's is for nominal variables so it's not a proper test in this case. Wilcoxon signed rank is more appropriate.
Penguin_Knight's user avatar

Only top scored, non community-wiki answers of a minimum length are eligible