I have a question about comparing the means of two variables that have a different meaning but a (possibly) similar ways of measuring them. To give an example:

Let's say I am conducting a study in the social sciences in which I ask my participants about their experiences of "Anger" and "Anxiety" while coping with a difficult situation. Each participant answers to both questions on a 7-point Likert-Scale.

Now my question(s):

  • Can I compare the means of the two variables? (e.g., "Do my participants rather experience Anger or Anxiety in the given situation?")
  • What statistical procedure is appropriate?
  • Is the result actually meaningful?

The answers to the first two question might be "Yes, do a paired-sample t-test or a repeated measures ANOVA." However, the two variables are not equivalent content-wise. And in my opinion the comparison of means between two such variables is not sensible unless it can be assumed that the variable share the same scale.

So while I think that the question is possible to answer from a "strictly statistical point", I wonder if interpreting the results would actually be meaningful. From my point of view "measuring" the two variables by the same Likert-Scale is not sufficient to make the comparison meaningful but I wonder if I am discarding it too readily.

| cite | improve this question | | | | |
  • $\begingroup$ There are paired tests that do not assume the same scale (eg rank-based); you could also re-scale/standardize the data. But - can't they experience both (could these categories be non-independent - and this might be what you are indirectly referring to)? Also, is 'situation' a factor? $\endgroup$ – katya Nov 11 '14 at 17:14
  • $\begingroup$ "Situation" is not a factor and assumed "constant". Participants can experience both, yes, maybe I should have said "Which feeling occurs more strongly/frequently?". I'll have a look into the rank-based tests, but intuitively I'd say that the problem of interpreting the results probably remains (being only vaguely familiar with non-parametric procedures). Thanks so far! $\endgroup$ – SimonG Nov 11 '14 at 18:13
  • $\begingroup$ rank-based approach would address "Which feeling occurs more strongly?" But is your question confounded by the fact that one is always going to be a little stronger (or easier shared in a survey;), thus the difference in scale? Yet this is something you can no longer address at the analysis stage, once the survey, responding to a known scale, has been completed. $\endgroup$ – katya Nov 11 '14 at 18:24
  • $\begingroup$ Yes, that was my initial concern. Thank you for commenting! If you summarize your point and post it as an answer, I will mark it as "accepted". $\endgroup$ – SimonG Nov 12 '14 at 10:38

Rank-based approaches (e.g. Wilcoxon paired test) would address "Which feeling occurs more strongly?" even in the presence of difference in scales. But the question could be confounded by the fact that one variable is going to be a little stronger - more willingly shared in a survey. Yet this can no longer be addressed at the analysis stage, once the survey, where participants were responding to a known scale, has been completed.

| cite | improve this answer | | | | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.