# McNemar test or Two proportion z-test?

Consider an experiment where a group of 200 people were asked before and after what is their preference, with possible outcomes being A and B. The research question is: Is there a difference between rate of change of preference from A to B and rate of change of preference from B to A?

Consider this table, where rows represent preferences before the experiment and columns represent preferences after the experiment:

Before/After A B
A 150 20
B 20 10

I could do a McNemar test to compare a proportion of preference A before the experiment to a proportion of preference A after the experiment, with non significant result, as frequencies 20 and 20 are the same (McNemar test takes into consideration just the discordant pairs). However, that is not consistent with the research question. Let me think out loud: "Out of 170 people, who preferred A before the experiment, 20 people changed their preference to B, which is 20/170x100=11,8 %. Out of 30 people, who preferred B before the experiment, 20 person changed their preference to A, which is 20/30x100=66,7 %. The rate of change of preference is 11,8 % for A->B and 66,7 % for B->A." These two rates are clearly different and I think it may be tested using two proportions z-test, with first population being people who prefer A before the experiment and second population being people who prefer B before the experiment. Then the group of 170 people can be considered as random sample from the first population and the group of 30 people can be considered as random sample from the second population. Then comparison of proportions 0,118 and 0,667 using z-test for two proportions answers my research question, with the result that the difference is statistically significant (z=-6,93, p=0,000).

Is that conceptually correct? Does it make sense? Because the data are paired in a sense that the same group of people were measured before and after the experiment and for paired nominal data McNemar is usualy used. However, with this specific research question I believe that two proportion z-test is appropriate.

The data is not paired because a participant chooses either A or B at their original preference. Compare with testing whether a patient has a disease with two different procedures A and B.

So McNemar's test is not applicable to begin with.

You can do a test for equality in proportions as you suggest. Alternatively, you can look at the odds ratio of the probabilities to change preference.

library("broom")
library("emmeans")

first_choice <- c("A", "B")
participants <- c(170, 30)
switch_choice <- c(20, 20)


Test if the probabilities to switch preference are equal:

tidy(prop.test(switch_choice,
participants))
#> # A tibble: 1 × 9
#>   estimate1 estimate2 statistic  p.value parameter conf.low conf.high method
#>       <dbl>     <dbl>     <dbl>    <dbl>     <dbl>    <dbl>     <dbl> <chr>
#> 1     0.118     0.667      44.7 2.33e-11         1   -0.744    -0.354 2-sample …
#> # … with 1 more variable: alternative <chr>


Estimate the odds ratio of the switch probabilities:

fit <- glm(
cbind(switch_choice, participants - switch_choice) ~ first_choice,
family = binomial
)

emmeans(
fit, pairwise ~ first_choice,
type = "response"
)
#> $$emmeans #> first_choice prob SE df asymp.LCL asymp.UCL #> A 0.118 0.0247 Inf 0.0772 0.175 #> B 0.667 0.0861 Inf 0.4835 0.810 #> #> Confidence level used: 0.95 #> Intervals are back-transformed from the logit scale #> #>$$contrasts
#>  contrast odds.ratio     SE  df null z.ratio p.value
#>  A / B        0.0667 0.0303 Inf    1  -5.957  <.0001
#>
#> Tests are performed on the log odds ratio scale


Created on 2022-06-18 by the reprex package (v2.0.1)