I have obtained two datasets. The pre where users were designated by a unique ID were asked to state what sources they used to make their decisions. Their decisions involve making a binary choice of deciding t0 assign their client a test or not ( they have a choice of 50 tests). At the end they also provide their client ID.
Next, I have the post data set. In this data set we have given them a new guidebook as a source they can chose to use to aid their decision making. The rest of the dataset is the same as the pre.
In the pre we only got participation from five users. These five users had an average 12 clients they could assign tests to. In the post we got participation from 12 users and had about 8 clients they could assign tests to. I have four users who were participated in both.
The research question is, did overall tests decrease after the implementation?
For most I have to use Wilcoxn to test the differences because normality is extremely violated. so I guess my question is what are some other ways to approach this? I think my next try will involve pulling random equal samples from both pre/post users to understand the difference but I'm not sure that's quite right.