Questions tagged [mcnemar-test]

A test for dependent categorical data, such as yes/no data before & after a treatment. The McNemar-Bowker test generalizes McNemar's test for $k>2$ categories. The tests assess whether the population proportions in each pair of corresponding symmetric cells are equal.

Filter by
Sorted by
Tagged with
1
vote
1answer
29 views

If we use McNemar's for single case data, is pseudoreplication an issue?

I have a query about McNemar's test. As far as I understand, the technique assumes that cells are independent. It tends to be used with group studies whereby counts refer to individuals, for example a ...
0
votes
0answers
29 views

P-values for sensitivity/specificity when only improvement is possible

Imagine two diagnostic tests in a single population at risk of the disease, one of which is always "more strict" than the other, meaning that if test 1 is positive, test 2 will be positive ...
0
votes
0answers
13 views

Can McNemar's Test say that Precision of one classification algorithm is better than the other?

Imagine this scenario: While presenting a new classification algo/model to my client, I asked him to run his existing algorithm (Model_Old) on 1000 objects and give me the Precision metrics, that is ...
0
votes
0answers
16 views

Statistical test: Independent categorical data with empty variables

I have 2 independent groups with categorical data. e.g. COPD GOLD classification [none, I, II, III, IV]. In group 2 I have zero GOLD II and zero gold IV patients. Which statistics are possible with ...
1
vote
0answers
13 views

Comparing responses between groups and conditions

I have a group of subjects that were tested under 2 visual stimuli. Each stimuli was presented in 2 directions (the stimulus is a flow of objects, once left to right and then right to left). And the ...
2
votes
1answer
256 views

Which test to use? Repeated measures with binary outcomes

I would like your opinion about which analyses to use for the research question. I've got a repeated measures with binary data. N = 20 H0 = there is no difference between pre and post measurement ...
0
votes
0answers
15 views

what metrics can i use to compare my two classifiers?

all, i have two classifiers (xgboost and light gradient boosting) to predict if yes cancer or not. when i use roc_auc as my scoring method i get xgboost as 0.75 and light gradient boosting as 0.76. ...
0
votes
0answers
10 views

Testing the difference in sensitivities between two lab tests - with part overlapping populations

I have two rapid diagnostic tests for which I test their sensitivity (+/-) in detecting cases against the gold standard (ELISA, also +/-). I now want to test whether the tests perform significantly ...
0
votes
0answers
12 views

Using R. Two groups (control, intervention) with one outcome as yes/no(nominal) measure at two different times (before and after). What test?

I would not post this question if I haven't exhausted researching this very simple question. I have two groups. (intervention) and (control). We measured an outcome variable (let's call it X) as yes ...
0
votes
1answer
22 views

Why can the null and alternate hypotheses of McNemar's test be written in terms of dependance/independance of tests?

Consider a study to examine whether food frequency questionnaires and three-day food diaries are equally likely to label a women as consuming less than the RDA of calcium. Because calcium intake can ...
0
votes
0answers
18 views

Can I test the difference between two chi square statistics for significance

I'm working on a (currently) hypothetical problem that relates to monitoring and evaluating people's participation in recycling. The test statistic is the participation rate (the proportion of ...
0
votes
2answers
170 views

What test to use to compare two distributions (from the same group of people)?

I have a data that consists of some "after treatment" and "before treatment" observations from some individuals. Specifically the data looks like this: ...
1
vote
0answers
12 views

Chi2 versus other tests for dependent samples (same observer different time)

I have come across a problem at deciding which hypothesis test I should use. I have personal previous research (great part of which is lost); of it, I have all the n and the proportions for each ...
3
votes
1answer
51 views

Is McNemar's Test Appropriate Here?

I am applying two diagnostic tests with dichotomous outcomes (condition present yes/no) to a sample of patient data. Both take into account a range of vital parameters and produce a test result, which ...
2
votes
1answer
423 views

Compare classification performance of two heuristics

I want to compare the classification performance of two heuristics: $h_0$ and $h_1$. I have around 700 samples for which I know the output class $c$ and the results of the two heuristics $c_{\textit{...
0
votes
0answers
23 views

What Statistical Analysis test do I run for Pre/post test?

Hello I am conducting a study where I examine patients who have not received a flu vaccination. In all I have 45 participants all who have not received the vaccine. I implemented and educational ...
2
votes
1answer
335 views

Is there a Bowker-like test to assess the symmetry of squared contingency tables with small expected values and a small number of observations?

I ask this question because Bowker test (McNemar extension for $k\times k$ tables with $k>2$) requires a large number of observations for fitting well and also requires (as a rule of thumb) that ...
1
vote
1answer
26 views

How to Test Sampling Methods

Suppose I have a population, $P$, of 100 individuals and I am devising 3 algorithmic methods to sample 20 people from said population: $A$, $B$, and $C$. We know that $C$ is a random, unbiased ...
2
votes
1answer
85 views

Chi Square test or McNemar test?

I've got a set of 1000 patients and I am measuring two different blood values (CRP and IL6) and choosing a categorical cutoff values for both (CRP>10 and IL6 >2). I now would like to know if there is ...
0
votes
0answers
27 views

McNemar's chi-square test for comparing two classifier algorithms

I want to compare the classification performance of Random forest with variable selection algorithm (method A) and Random forest only (method B). The original dataset has 100 variables. After using ...
0
votes
0answers
22 views

The McNemar exact conditional test: should the p-values be uniform under the null?

I have two classification models, $C_1$ and $C_2$, and I want to do a hypothesis test to see if $C_1$ is significantly better than $C_2$. I am interested in a one-sided hypothesis test. Thus I have ...
2
votes
1answer
293 views

Test if treatment effect is different between samples

I want to determine if there is significant variation in the effect of a treatment amongst my replicates. The treatment effect is measured as a proportion. Let me describe: I study heart defects (in ...
0
votes
0answers
41 views

Another solution than McNemar test for bilateral asymmetry on binary data?

Let's say I studied the shoulders of 10 individuals, and I scored the presence/absence of a criteria (ex: arthrosis, presence=0;absence=1) on the left side and the right side. I need to test the ...
1
vote
1answer
34 views

Statistical test for difference-hypothesis with dichotomous dependent and independent variables

I need your help. I'm designing a study for the following problem. There is a feature A (implemented by many software from my specific context) and I'm 100% sure that if I ask the users they will ...
1
vote
2answers
73 views

What statistical test would I use to test that both data sources are matching [closed]

Suppose I am only working with two datasets, Data Source 1 with columns Customer ID and ...
0
votes
1answer
32 views

Single Group Pre-Post McNemar Table Set Up

I am looking to evaluate if the change in proportion for a group changed pre-to-post intervention. Essentially, is the ~7% increase from pre (~2%) to post (~9%) significant. I am running the McNemar ...
0
votes
1answer
29 views

Chi-square based tests tending to zero for large sample sizes [duplicate]

I think I have some misconception about chi-square based tests such as Pearson's test or McNemar's test. The way the tests are defined, the test statistics produced by the formulas are not stable ...
0
votes
0answers
251 views

McNemar or Cochran's Q for evaluating multiclass classifiers?

full disclosure: I did a semi-cross post of this question due to low traffic. Once I get an answer on any of the two questions, I will link the answer back to the respective other. tl;dr For ...
2
votes
1answer
1k views

One-sided McNemar's test

In R, the function mcnemar.test has the following example: ...
4
votes
1answer
66 views

Can I perform a McNemar's test with only proportion and denominator information?

I have the following count and denominator information. The 630 subjects are the same people over two different periods of time. Can I conduct a McNemar's test to compare the dependent proportion ...
1
vote
1answer
392 views

Paired binary data with a zero cell

How do you analyse and obtain a p-value from a 2*2 table examining a binary outcome that is paired, when one cell has zero events? To clarify, I was looking at the success rate of obtaining an ...
0
votes
1answer
62 views

Conditional Exact Logistic Regression

I have paired data for two tests. I would like to say that Test A is more sensitive than Best B, but not sure if I'm using the correct methods or if my data support it. I know McNemar's Exact test can ...
1
vote
0answers
29 views

Hypothesis testing with two dependent multilevel categorical variables [closed]

I was presented with a requirement of comparing two sources of diagnoses on the same subjects, resulting in two 4-level categorical variables. I offered to calculate Cohen's Kappa, but the interested ...
1
vote
1answer
137 views

Comparison of proportions using McNemar's Chi-Square using Pre- and Post- data for two disease conditions

This question follows-up on the post: What is the difference between McNemar's test and the chi-squared test, and how do you know when to use each? I am examining a pre- and post- intervention data ...
1
vote
1answer
89 views

Multiple Observations per Subject per Time-point analysis?

I am having a difficult time finding an appropriate test for my analysis. Please help - In my study I have 130 participants who were tested with a binary outcome (pass [1] /fail [0]) before and ...
0
votes
0answers
72 views

comparing sensitivities

We want to compare two sampling methods. All patients will go through a DNA test for diagnosis(positive or negative). Then samples are taken to identify the determine the type of bacteria and ...
0
votes
1answer
98 views

Comparing proportions from overlapping samples

I'm looking to compare the read rate of an email newsletter with the read rate of another newsletter. However, many (not all) of the people who receive one newsletter also receive the other. What ...
1
vote
0answers
571 views

How do I get an odds ratio with McNemar's test?

I have pre and post test data evaluating the effectiveness of a lesson about how to help friends who are being bullied. I am trying to see if more students know how to help a friend being bullied ...
9
votes
4answers
4k views

Fisher exact test on paired data

Given $40$ cases with lung cancer and $40$ matched controls (without lung cancer) (matching based on age, sex, etc.). To try to find evidence between the effect of smoking on lung cancer, I used ...
2
votes
3answers
177 views

Hypothesis test for intervention with class imbalance before intervention. Alternative to McNemar

I want to assess an intervention to cure a disease. An example is described by the following matrix: ...
8
votes
2answers
7k views

Importance of McNemar test in caret::confusionMatrix

There are many metrics to evaluate the performance of predictive model. Many of these appear relatively straightforward to me (e.g. Accuracy, Kappa, AUC-ROC, etc.) but I am uncertain regarding the ...
1
vote
1answer
39 views

Use of Chi Square for only semi-independent sample?

I ran a study for my Master's thesis that counted the number of people who attended different courses that were advertised in different ways, and how many passed/failed each. I used a Chi Square ...
0
votes
1answer
1k views

Mcnemar's test p-value output in R confusion matrix

I am using the R caret package to build a randomforest classifier model for plant data. The dataset has 7 variables - all numeric which are being used to predict if a plant will "grow" or "not grow"....
0
votes
2answers
151 views

How to Control Familywise Error Rate in a series of Planned McNemar Comparisons

I have paired binary responses from same subjects under 9 different conditions. I conducted Cochran's Q which indicated that there significantly different pairs exist. I want to follow up but NOT ...
6
votes
2answers
923 views

Can Cramér's V be used an effect size measure for McNemar's test?

I'm using McNemar's exact test (with R's "mcnemar.exact" function). I know that giving the odds ratio or proportion would be an adequate measure of effect size (Effect size of McNemar's Test), but ...
0
votes
1answer
205 views

Relationship between statistical models: mcnemar test vs logistic regression

I always find useful and reliable answers in this forum and I hope this pattern will not change this time. I was spending some time to check whether an intervention program had any effect on an ...
1
vote
2answers
77 views

Most appropriate test to assess difference between categorical variables? Chi2 or Stuart-Maxwell?

I am trying to determine the most appropriate test for parts of my dataset. I have been using a $\chi^2$ test, but I am now realising it might not be the most appropriate. I am using Stata. I have a ...
30
votes
4answers
46k views

What is the difference between McNemar's test and the chi-squared test, and how do you know when to use each?

I have tried reading up on different sources, but I am still not clear what test would be the appropriate in my case. There are three different questions I am asking about my dataset: The subjects ...
1
vote
1answer
457 views

Sample size and number of events limitations for McNemar's test

What are sample size and number of events low limits for a proper application of McNemar's test with continuity correction? When must I use McNemar's exact test?
5
votes
2answers
2k views

Compare a diagnostic test to gold standard

I have a sample of patients who underwent 2 diagnostic tests, one of which is the gold standard. I've been asked to test if the diagnosis from the experimental test is different than the one from the ...