Questions tagged [mcnemar-test]
A test for dependent categorical data, such as yes/no data before & after a treatment. The McNemar-Bowker test generalizes McNemar's test for $k>2$ categories. The tests assess whether the population proportions in each pair of corresponding symmetric cells are equal.
114
questions
1
vote
0answers
58 views
Comparing the performance of random and weighted-random classifiers
I have a set of 300 observations of known class and there are 20 classes in total.
I want to compare the performance of a random classifier with a weighted-random classifier at classifying the ...
0
votes
1answer
97 views
Bayes Factors for McNemar Test
I am currently analyzing data from a within-subjects study in which all participants completed a standard and experimental task.
Responses were binary (pass = 1; fail = 0).
I believe that the ...
2
votes
1answer
223 views
What is the Bayesian counterpart to the McNemar test (of marginal homogeneity)?
In analogy to a similar post on continuous data, I ask the following. In paired discrete data, we test the frequentist hypothesis of equivalent marginal distributions of two discrete variables $X$ and ...
2
votes
1answer
1k views
McNemar test with multiple scores for the same subject
I am having a statistical issue:
subjects receive a test with machine A and machine B.
3 experts classify whether the results of machine A and machine B are acceptable (1) or not (0).
From my ...
1
vote
0answers
442 views
comparing classifers using McNemar Test
When comparing performance of two binary classifiers using McNemar test, should the two confusion matrices of the models be based on the training set, the validation set or even a second validation ...
3
votes
0answers
1k views
Chi-square test vs. McNemar's test - which test is right for this highly usual situation?
Friends,
Here is, I would assume, a fairly common situation: I would like to know which test to use to assess whether the tendency to answer "Yes" on question 1 of a questionnaire is associated with ...
3
votes
0answers
453 views
Pearson's chi-squared test assumption: independence of observations
I am unsure of how to establish (or interpret) the independence condition of the chi-squared test in (for instance) the following situation.
Consider this 2x2 contingency table representing the ...
2
votes
1answer
2k views
How to calculate information included in R's confusion matrix
I want know the formulas for the information highlighted in the confusion matrix in the picture below:
2
votes
1answer
217 views
Which test to use? Repeated measures with binary outcomes
I would like your opinion about which analyses to use for the research question.
I've got a repeated measures with binary data. N = 20
H0 = there is no difference between pre and post measurement
...
2
votes
1answer
286 views
Test if treatment effect is different between samples
I want to determine if there is significant variation in the effect of a treatment amongst my replicates. The treatment effect is measured as a proportion.
Let me describe:
I study heart defects (in ...
1
vote
1answer
686 views
mcNemar's Test how to compare two proportions in dependent sample
I would like to compare the false positive rates (1 - specificity) from two different tests within the same sample. The false positive rate is calculated by comparing the test values against the ...
1
vote
1answer
93 views
Site interaction with McNemar tests [closed]
I have some clinical diagnostic data from three sites that I have been looking at performance against a reference standard using the McNemar test.
How would I set up the analysis to investigate if ...
1
vote
2answers
2k views
Which test to choose when Wilcoxon test and McNemar's test are different?
I have paired data.The response variable is categorical with levels 1-5.
And I have used both McNemar's test and Wilcoxon rank sign test and I get different p-values. For McNemar's test a p-value of 0....
2
votes
2answers
909 views
McNemar's test p-value NA
I have paired data from 9 individuals. The response variable is ordinal with levels 1-5. As the sample is very small and the data are categorical, I thought the appropriate test was McNemar's test. ...
1
vote
0answers
1k views
Alternatives to McNemar-Bowker test when symmetrical cells have zero count in contingency matrix
I need to test the difference between pre- and post- intervention over a 4-grades parameter computed on 57 samples. The contingency matrix on which I need to compute a McNemar test for paired data (...
1
vote
1answer
152 views
Appropriate statistic for testing similarity of two paired binary datasets (of glaciovolcanism)
I am studying the ice-covered volcanoes of the world. I have developed three methods that use existing datasets to determine whether a particular volcano is icy or not, and I applied all three methods ...
1
vote
1answer
62 views
Statistically significance test to compare a ML classifier to a group of experts?
I am working on a medical data set of 60 test samples and I have the decisions of 45 medical experts for them over 5 diagnostic classes. How should I do hypothesis testing to show that the classifier ...
1
vote
0answers
160 views
Test for paired data on categorical variable with >2 levels
I am looking for the appropriate statistical test to compare frequencies over a categorical variable with >2 classes, for paired data (in practice, each individual has been labelled A, B, C or D ...
3
votes
1answer
1k views
McNemar's test vs chi-squared test for before & after data with different people from the same population
I have 2 proportions I want to compare. They are from the same population but at different time periods, in between which there was an intervention.
H0: Proportion in Period#1 = Proportion in Period#...
2
votes
1answer
267 views
McNemar test comparing two proportions in a crossover trial
In a crossover trial involving 50 patients, two dialysis treatment schedules were compared.Ten patients complained of vomiting on both schedules, 30 on neither, nine on treatment A only and one on ...
3
votes
1answer
708 views
Why Yates correction in McNemar's Test subtract 1?
McNemar's test statistic is given by:
Yates's correction defines something like this:
However, the correction for the McNemar's test statistic requires 1 (not 0.5)...
1
vote
0answers
119 views
Comparing binomial outcomes from prediction algorithms in R
I've written a simple model to predict the outcome of soccer games based on some input. The outcome is either "H","D",or "A", short for home team, away team, or draw. I'm comparing the results of this ...
2
votes
0answers
391 views
Can Cochran's Q test take a 3 level nominal variable as the response (instead of 2)?
I conducted a study using within-subjects (repeated measure) design for collecting nominal data. I have been reading a lot closet I get for a suitable test is Cochran's Q test since I have repeated ...
5
votes
3answers
645 views
Why aren't the diagonal counts used in McNemar's test?
In reviewing McNemar's test, I see that the diagonal counts aren't used in computing the test statistic. However, since the test is testing the null that the row and column marginals are equal, ...
1
vote
0answers
46 views
Can I sum up my count data instead of performing some repeated measures analysis?
I want to analyze establishment of plants in created gaps (plots):
I have 10 treated plots, which are divided into an outer section and an inner one and I have been looking at these plots over a ...
1
vote
0answers
1k views
Statistical test for paired data in an r x c contingency table
To perform statistical test on paired data when we have two dichotomous nominal variables (i.e., in a 2 x 2 contingency table) we can use the McNemar's test. When ...
1
vote
1answer
315 views
Compare classification performance of two heuristics
I want to compare the classification performance of two heuristics: $h_0$ and $h_1$. I have around 700 samples for which I know the output class $c$ and the results of the two heuristics $c_{\textit{...
1
vote
0answers
29 views
Self-selected groups categorical breakdown test
I have a population of people, and one group who was self-selected (group A), and the remainder of the population (group B), and each group is broken into a group of n nominal categories (categories ...
2
votes
1answer
532 views
Analyzing multiple dependent pairs of 2x2 contingency tables
In advance, I ask forgiveness for my statistical naïvety.
Suppose I have experimental data from pre- and post-treatment, and this data is split according to two binary categories like so:
\begin{...
2
votes
1answer
310 views
Is there a Bowker-like test to assess the symmetry of squared contingency tables with small expected values and a small number of observations?
I ask this question because Bowker test (McNemar extension for $k\times k$ tables with $k>2$) requires a large number of observations for fitting well and also requires (as a rule of thumb) that ...
9
votes
4answers
4k views
Fisher exact test on paired data
Given $40$ cases with lung cancer and $40$ matched controls (without lung cancer) (matching based on age, sex, etc.). To try to find evidence between the effect of smoking on lung cancer, I used ...
1
vote
0answers
79 views
dependent samples, binary data, multiple tests?
In each of my five experiments I run several trials which give me a certain success rate (e.g. 10 successful trials out of 30 tested), then a drug is applied and the same trials are run to give a ...
3
votes
1answer
130 views
McNemar's Test for repeated sex ratio data
I have proportion data (sex ratio of a dioecious plant) for a population that was measured three times. I would like to know if a McNemar's Test is the most appropriate analysis to answer the question:...
4
votes
1answer
527 views
Contingency table for gene groups
I have two lists of genes as follows:
DEList has 282 gene names
AllList has 32805 gene names
...
3
votes
0answers
130 views
Does conditional logistic regression effectively “throw away” concordant pairs like McNemar's test?
We are interested in analyzing pre-post data from a randomized control trial. The outcome is binary and I am trying to explain the heuristics of the model formulation for conditional logistic ...
1
vote
0answers
19 views
Analysis of possibly correlated small sample data
I'm doing an experiment where I'm imaging 3 groups of 5 rabbits (total = 15 rabbits) using 4 different machines at 4 different anatomical locations to see if a radio-opaque material is visible in all ...
7
votes
2answers
6k views
Importance of McNemar test in caret::confusionMatrix
There are many metrics to evaluate the performance of predictive model. Many of these appear relatively straightforward to me (e.g. Accuracy, Kappa, AUC-ROC, etc.) but I am uncertain regarding the ...
1
vote
1answer
981 views
McNemar or chi-square - test for group differences paired and unpaired data
I understand that the difference between a chi-square test of homogeneity and the McNemar test is that the chi square is used for unpaired data and the McNemar is used for paire data.
Here's my ...
2
votes
2answers
525 views
How to get random classification to assess the performance of classifier with McNemar test?
I'm trying to replicate a study where the author used the McNemar test to assess the performance of classification compared to random classification. I have the original classifier and I'm using R to ...
0
votes
0answers
192 views
Two tails or one tail McNemar's test in a binary classification problem. Which one should i choose?
I'm using McNemar's test for compare two designed models in a binary classification problem. As you know we have two kinds of this test. ...
2
votes
1answer
134 views
Test for pairs of small integers
I have two classifiers, say $A$ and $B$ and I am performing 10$\times$10-fold cross-validation on them and record the number of instances correctly classified from the test set. Unfortunately, the ...
1
vote
0answers
279 views
Fisher's & McNemar's tests
I'm conducting a retrospective study to assess the efficacy of a new test for predicting sudden cardiac death (SCD). I also want to compare the accuracy of my new predictor with that of a conventional ...
2
votes
1answer
3k views
Cochran's Q & McNemar tests together
I have binomial outcome data for three groups that I am analyzing with Cochran's Q test. As discussed at this site, Cochran.
However, I have to repeat this seven different times because I am ...
4
votes
1answer
916 views
Proper analyses for 2x2 contingency tables
So I have multiple questions about the different types of analyses that can be used on a 2x2 contingency table and when different analyses should be used.
To open, I'll note that this stems from my ...
5
votes
2answers
486 views
Can McNemar's test be improved upon by adjustments for zeros like those in a sign test?
McNemar's test is a special example of the binomial sign test, but the ('vanilla') sign test suffers from bias due to ignoring differences equal to zero.
McNemar's test statistic is given by:
$\chi^{...
4
votes
2answers
974 views
Is the binomial sign test really the exact McNemar test?
I am calculating two different $p$ values for the McNemar test and what I believe is the corresponding exact test, even for combined counts larger than 25.
Take, for example the contingency table
<...
0
votes
0answers
227 views
McNemar - is it the right test for this example? Effect size?
I have two dependent groups that could have a disease before and after a treatment.
My sample size is 12214 subjects.
Before the treatment, 7 of them had the disease and after the treatment 14 of them ...
0
votes
1answer
6k views
Where does p-value come from in McNemar's Test?
I was using this online calculator to work out McNemar's Test on this data set:
I get the results:
The two-tailed P value equals 0.1859
Chi squared equals 1.750 with 1 degrees of freedom.
...
2
votes
0answers
220 views
Comparison of multiple diagnostic tests with paired data
Good Day All,
I am attempting to compare three diagnostic tests against a gold-standard and for simplicity sake, let us have all the outcomes be binary (disease/no disease).
In this study study, all ...
2
votes
0answers
254 views
Comparing proportions from the (somewhat) same sample
I’m looking for some help on testing whether there are statistical differences between proportions when the data is dependent. By dependent, I mean that all the data is from the same participants ...
