Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Which type of test is most appropriate for this comparative-descriptive study? I have two groups of subjects (250 in each). My goal is to determine if the two subject groups are different and if so, how. I'm using an online survey of 50 statements (embedded are 5 statements about 10 different possible factors). Subjects rate these statements on a Likert scale from "strongly agree" to "strongly disagree". Do I have to use a t-test or a logistic regression to determine categorical differences, and predictors?

share|improve this question
Are you interested in finding a general difference in agreement or are you interested in testing each individual question? How many levels in each scale? Do the questions test different things? Are they mutually exclusive (i.e. rankings that must all be different like a sort)? Please provide more details on what you intended in your study. – John Apr 6 '12 at 15:53

If you have a rating scale (what you're incorrectly calling a Likert scale) with a lot of levels, say about 10, then it's sort of OK to treat the values of the scale as interval. Small scales should not be treated this way. Nevertheless, it sounds like you're actually generating a Likert scale, which is the aggregate, usually sum, of the collections of ratings that you've gathered, and not any individual rating (your question confuses that issue). In that case it's usually fine to just treat it as interval data.

You could use a non-parametric test or parametric test. It's going to come down to whether you think some kind of model can be fit to the data and you can estimate parameters of it. Does the central limit theorem suggest the data are normally distributed and the resulting distributions in line with that? If so, a t-test is probably fine. If not then perhaps a simple non-parametric test of some kind. I'd probably prefer in your case to bootstrap a confidence interval of the effect.

Regarding logistic regression, perhaps you were considering it because, while the full scale can be treated as interval, ordinal logistic regression could be used on an individual rating regardless of whether there are a large or small number of levels and it's designed specifically for that kind of thing. There's a fairly nice ordinal package in R. You'd want the clm command, or maybe clmm command. A more specific question could help here.

share|improve this answer

The t-test is not appropriate for the data you have. With ordinal data or Likert scales, as in your case, you should use Wilcoxon's rank-sum test, sometimes also called the Mann–Whitney U test. Using this test, you can assess for each of the 50 statements if there is a statistically significant difference between the two groups.

This test should be available in your favorite statistical software. Moreover, it is not too hard to compute this test using a spreadsheet.

share|improve this answer
The questioner said logistic regression... assuming it was ordinal logistic regression then that is the appropriate method. – John Apr 6 '12 at 15:45

(I wanted to comment on lejohn's post, but I can's seem to find the option anywhere.)

Likert scales are almost always treated as interval level data. I'm guessing that, based on your original post, you will not be treating each item separately, but will be aggregating to create 10 subscales composed of 5 items each (using either the sum or the mean score). Unless these subscales have distributions that are extremely non-normal or have wildly heterogeneous variances between your subsamples, you are safe, and in my opinion advised, to use a t test (and perhaps a Bonferroni correction or similar).

share|improve this answer
I think you need some reputation to comment (which you should have enough now). Welcome to the site! – Andy W Apr 6 '12 at 15:48
just because lots of people treat likert scales as interval doesn't mean it's right... it's only defensible in certain circumsances – John Apr 6 '12 at 16:01
I assume those certain circumstances are those that you speak of above (i.e., when there are 10 or more levels). This is arbitrary and nonsensical. Why, when there are more scale points, should we be fine to treat the variable as interval than when the theoretical distances between the scale points are greater? From a psychometric perspective, there is plenty of evidence in favour of treating the scale points of a Likert scale as interval level data, and not only because "lots of people do it". A good start would be reading Rensis Likert's original (1935) article. – Jehu Apr 6 '12 at 23:11
It's more civil to set the threshold higher for calling an argument "nonsensical." Beyond that, there is a decades-long debate about the merits of treating an individual rating item as interval-level. But a Likert scale, which is a composite score calculated based on multiple rating items, is almost always treated as interval-level.… – rolando2 Apr 6 '12 at 23:36
A fair point, and my apologies to John. I agree with your comment, and sense that this is the case for the original poster (i.e., I don't believe his intention is to perform a test for each individual scale item). – Jehu Apr 6 '12 at 23:41

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.