What test to use to compare two distributions (from the same group of people)? I have a data that consists of some "after treatment" and "before treatment" observations from some individuals.
Specifically the data looks like this:
        A   B   C    D
before  10  15  4    0
after   12  12  2    3

A = not agree
B = slightly not agree
C = agree
D = definitely agree
Basically I have a group of individuals ($N=29$) who did a survey which consists of answering some questions before and after some intervention. My question is how do I test if there is difference in the "before" and "after" proportion of answering the question.
For example, for the "before" row, there are 10/29 people answers "not agree", but after the intervention, 12/29 people answers "not agree". 
I was thinking of the chi-square test, but it is definitely wrong to use chi-square since the samples of "before" and "after" are coming from the same group of individuals. McNemar test won't work since I have more than 2 categories. 
Could someone advise as to what test I can use?
 A: The test you are looking for is called a test of marginal homogeneity (the table you give are the two margins of a 4x4 cross-classification, and you want to test if these are equal, hence the name). There is at least one R-packages for the Stuart-Maxwell test, which is a Wald test. A bit more accurate for small samples may be a likelihood ratio test using maximum likelihood estimators, and there are packages by Joe Lang, Antonio Forcina et al., or Wicher Bergsma (package cmm), however these can do much more and just marginal homogeneity so you for a quick test maybe search for the Stuart-Maxwell one.
A: I can think of four paired-tests that you could use, for better or worse, in this scenario:


*

*Force the data to work with McNemar's test by combining categories A+B and C+D. This is far from ideal, as you will lose quite a bit of power.

*Convert A,B,C,D to 1,2,3,4 and use the sign test. If there are many unsigned pairs (ie, for many respondents the level of agreement does not change after treatment), this is not a good choice. 

*Convert A,B,C,D to 1,2,3,4 and use the Wilcoxon signed-rank test. This will be useful if you are interested in the direction of change of agreement before and after, rather than if you are interested in just whether or not a change has occurred. 

*Use the McNemar-Bowker test (no Wikipedia page!) which is an extension of McNemar's test for 3 or more response categories. This only tells you if there is a difference after treatment, not if this difference tends to be in a specific direction (unlike previous options). It ignores the ordinality of your groups.


Check the assumptions for each test are (approximately) met before using them. And, ideally, choose a test a-priori! Don't try all four then pick the one which suits you best.
