Which statistical test to use? Paired-sample or independent-sample?

Question

I have 1 subject rating 40 participants on the big five personality traits and other variables. these 40 participants are in 2 conditions (20 each). I want to compare whether there is a difference between how the subject rates the two conditions. I know it sounds like paired samples, but doesn't paired samples compare the difference of 2 answers for the same participant? I have one subject giving 40 answers.

Also practically, how should I construct my data set? Paired samples cannot test the difference between conditions. I would have to make new variables for condition 2 and then compare these. In SPSS, however, won't it look as if participant 1 and participant 20 are the same person, since their columns would be next to each other on the data set.
Any tips on what analysis to use? I have to hand in the study in a few days, so fast advice would be appreciated!

Answer 1

If each of these 40 participants only ever see one condition, then you have a traditional between-subject design and would use independent samples tests. The way you describe your data formatting problem in SPSS at the end suggests that this in fact your situation.

If each participant undergoes both conditions, then observations under each condition are not independent anymore. You should then use a paired sample test. Note that it could even be the case if the participants are different but you have some ways to match them (say you select them in pairs based on age, gender, etc.)

Usually, the relevant question is whether the data comes from the same participants or not and that's typically the way these tests are taught but there is a broader notion of “independence” lurking underneath. If the participants were situations or objects or anything else that your rater had to evaluate, the logic would remain the same.

On the other hand, the fact that there is one rater in common does not introduce any dependence as it is constant across the design. It's a person instead of a piece of equipment but logically it is not much different than using the same room or the same pen for the whole experiment. Should the color of the pen have some sort of effect on the response of interest, you would not be able to generalize your results to other pens but statistical inferences about your manipulation are not threatened as such.