1
$\begingroup$

I asked 12 participants in a small experiment to order 5 distorted webpages from the least to most trustworthy (1 - 5). I want to know the best way to test how similar participants answers are to a set of expert's answers.

Previously I have provided Cronbach's Alpha scores of internal consistency for the participants' answers, but this does not tell me how similar they are to the Expert answers, just how consistent they are with each other.

If anyone could provide me with a name of a suitable test it would be very much appreciated.

Expert Answers

1 2 3 4 5

Participant Answers

1 2 3 4 5 
2 1 3 4 5
1 2 5 4 3
2 1 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 3 2 4 5
1 2 4 3 5
1 2 3 5 4
2 1 3 4 5
1 2 3 4 5
|cite|improve this question
$\endgroup$
  • 1
    $\begingroup$ If you just want a distance measure, you could calculate Gower's distance between the participants and the expert answer. But you mention a test, which implies that you want to perform a formal hypothesis test. If that is true, what hypothesis do you want to test, exactly? $\endgroup$ – COOLSerdash Sep 4 at 9:43
  • $\begingroup$ Thanks, I've never heard of Gower's Distance. I took a quick look and it seems like it would be suitable. I could report the Cronbach's Alpha for the similarity of the Participant Answers, and then maybe work out and report the Gower's Distance on the mean of each column above compared to the Expert Answers. I've never really thought of this as a formal Hypothesis test, (as it was just a pre-test) but I suppose it is. I suppose the hypothesis would be: H0: “Participant Answers will be similar to Expert Answers" --- H1: “Participant Answers will not be similar to Expert Answers" $\endgroup$ – Deepend Sep 4 at 11:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.