I performed 3 experiments, each experiment tested a different system, and each experiment involved a distinct group of subjects. In total there where 25 participants (7 for the first system, 7 for the second, 11 for the third). Subjects were asked to identify a stimulus (i.e. a vibration provided by a haptic device).

Recognition was measured as 1 = correct identification, 0 = incorrect identification. Stimuli where repeated, each stimulus receive 0 or 1 as measurement of participants' responses to it.

The stimuli where different for each system. Still I want to test whether participants using one system performed better than the participants using other systems.

My goal is only to assess the statistical differences between the 3 groups. Which non-parametric analysis I have to perform for this case of between-subjects experimental design?

I was thinking to the Mann-Whitney-Wilcoxon test, but I am unsure if I am correct. Is there anyone who could suggest which is the right analysis and provide a R example?

  • $\begingroup$ 1. Can you clarify what "various stimuli in a repeated fashion" involves? It sounds like you have treatment ("system"), subject group and stimulus as variables, in which case I don't know if Friedman will be suitable. 2. Can you talk about your response variable? How is it measured? 3. What exactly did you test with the Shapiro-Wilk? I can't see you being able to perform a test that relates to any assumptions in the absence of a suitable model (and if you don't know which test to apply I don't see how you can have one). ... ctd $\endgroup$ – Glen_b Jul 7 '18 at 23:12
  • $\begingroup$ ctd... 4. Formally testing assumptions (even when done correctly) is often not helpful ... and using it to choose between tests impacts the properties of your subsequent tests, intervals etc - so your p-values in your analysis will be wrong, for example. 5. There are numerous questions that ask about nonparametric tests for repeated measures of various designs, some of which have answers that may have relevant information to you, so it would be important to try some searches. $\endgroup$ – Glen_b Jul 7 '18 at 23:15
  • $\begingroup$ 1. There were various stimuli to identify which were provided by the 3 systems (the stimuli were different for each system). 2. The response variable was measured after an identification task. It had two values, 1 = participants correctly identified the stimuli, 0 if they did not. 3. I tested the normality of my data. In any case I want to use a non-parametric analysis as in total I only have 25 participants, so ANOVA is not really well suited as far as I know. $\endgroup$ – L_T Jul 8 '18 at 17:49
  • $\begingroup$ 4. and 5. Thanks, what is your suggestion then? I guess that for my case the correct technique is to use three different comparisons between couple of groups using the mann whitney wilcoxon test. What do you think? $\endgroup$ – L_T Jul 8 '18 at 17:49
  • $\begingroup$ I don't follow your answer in 1, and so still don't understand the experiment properly (... there's danger in deciding on analysis after you have data, by the way). In 2. where you say "the stimuli" -- in particular the use of the plural here -- participants are identifying more than one stimulus at once? Or are your responses a single 0 or 1 each time? On 3. Why would you test normality of scores of 0 or 1? That can't be normal. On 4. can you explain what makes a nonparametric analysis better at small sample sizes?... $\endgroup$ – Glen_b Jul 9 '18 at 0:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.