# McNemar's test vs chi-squared test for before & after data with different people from the same population

I have 2 proportions I want to compare. They are from the same population but at different time periods, in between which there was an intervention.

• H0: Proportion in Period#1 = Proportion in Period#2
• H1: Proportion in Period#1 ≠ Proportion in Period#2

Would this call for a McNemar's test instead of chi-squared? If yes, how do we interpret the odds ratio?

• Do you have the same units measured in both proportions, or are they different units, but from the same population? Jun 24 '16 at 21:12
• they are people, same units. I have the absolute numbers. Jun 24 '16 at 21:16
• So you have the exact same people at both time points? Jun 24 '16 at 21:30
• they are not the exact same people (i couldnt know that, data is deidentified) but they are from the same geographical location taken original from a national sample. Jun 24 '16 at 21:36
• I have to clarify that the even though the people in each period are not the same, they do come from the same "stratum" of the population sample. I would assume that they are "new", unique, observations. In other words, I want to compare a proportion within a geographical area before and after, with the subjects not being the same ones at each timepoint. Jun 25 '16 at 14:32

## 1 Answer

Since you don't have the same people in both samples, you should use the chi-squared test. You would set it up like this:

    Period 1  Period 2
yes     n_y1      n_y2
no      n_n1      n_n2


where n_rp is the number of people with the given response in the specified time period. At this point, I usually tell people to use the $z$-test for the difference of two proportions because I think that is conceptually clearer for people, but the $z$-test is mathematically equivalent to the chi-squared test, it's just that software will often present the output differently. So you can use the chi-squared if you prefer.

An odds ratio will work well as an effect size measure for your case. You would form it as:
$$OR = \frac{\frac{n_{y2}}{n_{n2}}}{\frac{n_{y1}}{n_{n1}}} = \frac{n_{y2}n_{n1}}{n_{n2}n_{y1}}$$ The standard interpretation would apply. That is, the odds of 'yes' are OR times higher in period 2 than in period 1 (cf., Odds made simple).

McNemar's test is appropriate when you have the exact same units (people in your case) in both time periods. In that case, the setup is as follows:

             Period 2
Period 1      yes    no  prop1
yes    n_yy  n_yn   n_y.
no     n_ny  n_nn   n_n.
prop2  n_.y  n_.n


where n_rr is the number who responded the indicated way in each time period. In this case, you want to assess if the marginal proportions are the same, but you actually do that by running a binomial test (or chi-squared with some adaptations) on the n_yn and n_ny counts. For more on how that would work, see my answers here and here.