# What test to be used for comparing pre and post test results with categorical data?

In the past month(s) I had to get familiar with statistics in order to validate my results although my background is computer science/programming. I currently examine whether I can enhance user's skills with a tutorial. So I prepared a pre and a post test and measured various skills to compare them. The Wilcoxon Signed Rank test helped me to determine significance since my data was not normally distributed.

Now here is my current problem: I examine whether my subjects used a tactic within a (fact-finding search) task and whether the post results differ significantly from the pre test results. For each user, my data keeps track whether he applied (used) the tactic or not (not used).

UPDATE I want to conclude whether the intervention had a significant impact on my subjects' search skills, i.e., whether more users applied the (search) tactic in the post test.

My data looks as follows:

N = 58 users
0 = no
1 = yes

user pre  post
1    1    1
2    1    1
3    0    1
4    0    1
5    0    1
6    0    1
7    0    1
8    0    1
9    1    1
10   0    1
11   0    1
12   1    1
13   1    1
14   0    1
15   0    1
16   0    0
17   1    1
18   1    0
19   1    1
20   0    1
21   1    1
22   0    1
23   1    0
24   0    1
25   0    0
26   1    0
27   1    1
28   0    0
29   1    1
30   1    1
31   0    0
32   0    1
33   1    0
34   0    0
35   1    1
36   0    1
37   0    1
38   0    1
39   0    1
40   1    1
41   1    1
42   1    1
43   0    1
44   1    0
45   0    1
46   1    1
47   1    1
48   1    1
49   0    1
50   1    1
51   0    1
52   1    1
53   0    1
54   0    1
55   0    1
56   1    1
57   0    0
58   1    1


Summary:

                         Post
|Not used | Used|
Pre   |Not used  |6        |25   |
|Used      |5        |22   |


*UPDATE END *

I read a lot about significance tests, Chi-square, McNemar, Fisher’s exact test, ... but I am still not sure which test is appropriate or whether I can determine significance at all for that kind of data.

• Have a look at: stats.stackexchange.com/questions/3466/… – kjetil b halvorsen Dec 30 '18 at 14:27
• Thanks for the hint! My problem is I have only 1 group: the same users are measured twice. And I want to check whether the score differ. – Ioannis K. Dec 30 '18 at 15:26
• The table you have shown can be looked upon otherwise: Each one of the 58 users can have one of four transitions: not use -> not use; not use -> use; ... and so on. Can you show us that table also? (as an edit to the post) – kjetil b halvorsen Dec 31 '18 at 20:28
• Also, you could get some ideas from this post – kjetil b halvorsen Dec 31 '18 at 20:39
• Yes, I would go for the McNemar test. But read well the linked posts here about that test! – kjetil b halvorsen Jan 2 at 23:24