Which type of test is most appropriate for this comparative-descriptive study? I have two groups of subjects (250 in each). My goal is to determine if the two subject groups are different and if so, how. I'm using an online survey of 50 statements (embedded are 5 statements about 10 different possible factors). Subjects rate these statements on a Likert scale from "strongly agree" to "strongly disagree". Do I have to use a t-test or a logistic regression to determine categorical differences, and predictors?
- Anybody can ask a question
- Anybody can answer
- The best answers are voted up and rise to the top
If you have a rating scale (what you're incorrectly calling a Likert scale) with a lot of levels, say about 10, then it's sort of OK to treat the values of the scale as interval. Small scales should not be treated this way. Nevertheless, it sounds like you're actually generating a Likert scale, which is the aggregate, usually sum, of the collections of ratings that you've gathered, and not any individual rating (your question confuses that issue). In that case it's usually fine to just treat it as interval data.
You could use a non-parametric test or parametric test. It's going to come down to whether you think some kind of model can be fit to the data and you can estimate parameters of it. Does the central limit theorem suggest the data are normally distributed and the resulting distributions in line with that? If so, a t-test is probably fine. If not then perhaps a simple non-parametric test of some kind. I'd probably prefer in your case to bootstrap a confidence interval of the effect.
Regarding logistic regression, perhaps you were considering it because, while the full scale can be treated as interval, ordinal logistic regression could be used on an individual rating regardless of whether there are a large or small number of levels and it's designed specifically for that kind of thing. There's a fairly nice
The t-test is not appropriate for the data you have. With ordinal data or Likert scales, as in your case, you should use Wilcoxon's rank-sum test, sometimes also called the Mann–Whitney U test. Using this test, you can assess for each of the 50 statements if there is a statistically significant difference between the two groups.
This test should be available in your favorite statistical software. Moreover, it is not too hard to compute this test using a spreadsheet.
(I wanted to comment on lejohn's post, but I can's seem to find the option anywhere.)
Likert scales are almost always treated as interval level data. I'm guessing that, based on your original post, you will not be treating each item separately, but will be aggregating to create 10 subscales composed of 5 items each (using either the sum or the mean score). Unless these subscales have distributions that are extremely non-normal or have wildly heterogeneous variances between your subsamples, you are safe, and in my opinion advised, to use a t test (and perhaps a Bonferroni correction or similar).