0
$\begingroup$

This is a pretest-posttest design with a control group (no treatment between measurements) and an experimental group (treatment between measurements).

All measurements consist of the same rubric and provide ordinal data (1,2,3,4).

Tests for metric data like t-tests are not appropriate. Many websites suggest using the Mann-Whitney U test as a non-parametric test for independent groups, with the difference between pretest and posttest score, i.e, gain score. My problem with these suggestions is that with ordinal data calculations like differences are not possible.

What test would be appropriate for this pre-test/post-test design?

Edit 2: Calculating differences is not possible for ordinal data

Pitouille write about "calculate the difference between post and pre-test". In my understanding, it is not possible to determine the difference with ordinal data, as it is not defined at all. If I could calculate the difference, the data would have to be at least interval scaled.

The difference corresponds to the usual gain score, which I thought of first. But because of the above reasoning, it is not possible to form the gain score.

Hence my question is: How can I determine the effects in pretest-posttest groups if I cannot calculate differences?

Edit 1: Possible examples of data.

First to be mentioned, the study is in the planning phase and I have not yet collected any data. Each group is expected to have 15 to 20 participants.

Candidate Control(0) or Experiment(1) group? Pretest-Score Posttest-Score
A 0 1 1
B 0 1 2
C 0 2 2
D 1 2 3
E 1 2 2
F 1 3 4
$\endgroup$
3
  • $\begingroup$ To understand better your problem, it would be helpful if you can share examples of the data you have to deal with. $\endgroup$
    – Pitouille
    Sep 3, 2021 at 5:08
  • $\begingroup$ @Pitouille Thank you for your quick reply! If the examples attached are not sufficient, please let me know. $\endgroup$
    – Svana
    Sep 3, 2021 at 7:27
  • $\begingroup$ I feel your question is more about structuring your test... did I get it right? $\endgroup$
    – Pitouille
    Sep 3, 2021 at 12:48

1 Answer 1

0
$\begingroup$

I might miss the point... but at least it could contribute to push your question forward!

My understanding is that you are looking for the approach to conduct your test rather the test itself.

Supposing that the assumptions to conduct a Mann-Whitney U test are met, the approach would be to split your data in 2 samples (control and experiment) and for each candidate, calculate the difference between post and pre test.

Control

Canditate Post - Pre
A 0
B 1
C 0

Experiment

Canditate Post - Pre
D 1
E 0
F 1

... and then perform your test with you two independent samples.

EDIT:

So, I did miss the point :-)

There is study from Winter and Dodou that you can find in the stats literature that could be relevant to your problem: https://blog.minitab.com/en/adventures-in-statistics-2/best-way-to-analyze-likert-item-data-two-sample-t-test-versus-mann-whitney

You can also have a look at @Glen_b's answer: Mann Whitney or two tailed t-test

$\endgroup$
1
  • $\begingroup$ Formerly, that was my idea too. In the post with the question, I added my concern as it was too long for the comment. $\endgroup$
    – Svana
    Sep 3, 2021 at 16:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.