1
$\begingroup$

In the past month(s) I had to get familiar with statistics in order to validate my results although my background is computer science/programming. I currently examine whether I can enhance user's skills with a tutorial. So I prepared a pre and a post test and measured various skills to compare them. The Wilcoxon Signed Rank test helped me to determine significance since my data was not normally distributed.

Now here is my current problem: I examine whether my subjects used a tactic within a (fact-finding search) task and whether the post results differ significantly from the pre test results. For each user, my data keeps track whether he applied (used) the tactic or not (not used).

UPDATE I want to conclude whether the intervention had a significant impact on my subjects' search skills, i.e., whether more users applied the (search) tactic in the post test.

My data looks as follows:

N = 58 users
0 = no
1 = yes

user pre  post
1    1    1
2    1    1
3    0    1
4    0    1
5    0    1
6    0    1
7    0    1
8    0    1
9    1    1
10   0    1
11   0    1
12   1    1
13   1    1
14   0    1
15   0    1
16   0    0
17   1    1
18   1    0
19   1    1
20   0    1
21   1    1
22   0    1
23   1    0
24   0    1
25   0    0
26   1    0
27   1    1
28   0    0
29   1    1
30   1    1
31   0    0
32   0    1
33   1    0
34   0    0
35   1    1
36   0    1
37   0    1
38   0    1
39   0    1
40   1    1
41   1    1
42   1    1
43   0    1
44   1    0
45   0    1
46   1    1
47   1    1
48   1    1
49   0    1
50   1    1
51   0    1
52   1    1
53   0    1
54   0    1
55   0    1
56   1    1
57   0    0
58   1    1

Summary:

                         Post 
                 |Not used | Used|
Pre   |Not used  |6        |25   |
      |Used      |5        |22   |

*UPDATE END *

I read a lot about significance tests, Chi-square, McNemar, Fisher’s exact test, ... but I am still not sure which test is appropriate or whether I can determine significance at all for that kind of data.

$\endgroup$
  • 1
    $\begingroup$ Have a look at: stats.stackexchange.com/questions/3466/… $\endgroup$ – kjetil b halvorsen Dec 30 '18 at 14:27
  • $\begingroup$ Thanks for the hint! My problem is I have only 1 group: the same users are measured twice. And I want to check whether the score differ. $\endgroup$ – Ioannis K. Dec 30 '18 at 15:26
  • 2
    $\begingroup$ The table you have shown can be looked upon otherwise: Each one of the 58 users can have one of four transitions: not use -> not use; not use -> use; ... and so on. Can you show us that table also? (as an edit to the post) $\endgroup$ – kjetil b halvorsen Dec 31 '18 at 20:28
  • 1
    $\begingroup$ Also, you could get some ideas from this post $\endgroup$ – kjetil b halvorsen Dec 31 '18 at 20:39
  • 2
    $\begingroup$ Yes, I would go for the McNemar test. But read well the linked posts here about that test! $\endgroup$ – kjetil b halvorsen Jan 2 at 23:24
1
$\begingroup$

Go for a McNemar test it may give you the accuare results in your case. also you can compare the results from wilcoxon signed rank test i.e dependent test with McNemar test then you can conclude your result. https://rcompanion.org/handbook/H_05.html above link will tell you a path.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.