2
$\begingroup$

Could someone recommend me sources where I could find illustrated examples, solved by hand and/or using software, on how a Likert scale questionnaire (with more than 1 question, say 10, 20, etc. questions; where each question has items to rate, say 3,4, etc; where (0) means disagree and (1) agree or (1) disagree and (5) agree (I think this is a matter of taste?)) "interacts" with ANOVA and Kruskal-Wallis test.

I've seen many examples and plenty of exercises on books but they are always like

  • Provides context and some numerical data sets.

  • Use of ANOVA (one or two way or repeated) or Kruskal-Wallis.

  • and Results.

So far I've never encountered one that in the context considers a Likert-scale questionnaire, data, ANOVA or Kruskal-Wallis and results.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
+100
$\begingroup$

EDIT: Based on a critical edit to your question: Yeah, sums of Likert items do not have a Likert distribution. Thanks to the Central Limit Theorem they have an approximately normal distribution. Approximately normal data are pretty soundly in ANOVA's bailiwick, (the more items contributing to your outcome variable, the more comfortable with ANOVA you should be), though you will still need to make appropriate corrections for unequal variances between groups.

If the number of Likert-scale items contributing to your variable is small, you might want to use the Kruskal-Wallis test instead. There will be a small hit to statistical power (as compared with ANOVA) using Kruskal-Wallis with approximately normal data, but you should be able to use either.

The Kruskal-Wallis test assumes the outcome data measured across $C$ groups are measured continuously. I.e., from their original paper "If the samples come from identical continuous populations, and the $n_{i}$ are not too small, $H$ is distributed as $\chi^{2}(C-1)$…" ($H$ is the Kruskal-Wallis test statistic, and the $n_{i}$s are the sample sizes in each group.)

The same is true of the Mann-Whitney(-Wilcoxon) rank sum test between two samples—which the Kruskal-Wallis test is effectively a $C$ sample extension of—where Mann & Whitney's first sentence is "Let $x$ and $y$ be two random variables with continuous cumulative distribution functions $f$ and $g$."

Unfortunately, this means that the inferential properties based on the distribution of the $H$ test statistic may be invalid if applied to, for example, ordinal data of only a few values. While Kruskal & Wallis do give corrections for ties, which may arise in any continuous variable where $n>10^p$ where $p$ is the precision in number of significant digits, I suspect the test is unreliable for cases where all values are tied many times, as would be the case for Likert scale data with 5-ish or 7-ish values.

At the very least you may expect to find few illustrated examples published which violate an assumption of the test.


References
Kruskal, W. H., & Wallis, W. A. (1952). Use of ranks in one-criterion variance analysis. Journal of the American Statistical Association, 47(260), 583–621.

Mann, H. B., & Whitney, D. R. (1947). On A Test Of Whether One Of Two Random Variables Is Stochastically Larger Than The Other. Annals of Mathematical Statistics, 18, 50–60.

Improve this answer
$\endgroup$
25
  • 1
    $\begingroup$ @America That is what I am saying, yes. Sorry if that wasn't clear. :) (Also: welcome to CV!) $\endgroup$
    – Alexis
    May 15, 2020 at 21:57
  • 2
    $\begingroup$ @America ANOVA also has assumptions about continuous variables—in fact more stringent, since ANOVA assumes continuous normally distributed variables. I think for Likert outcomes/dependent variable, I might go to something like ordered logit regression, with group as a predicting factor variable. $\endgroup$
    – Alexis
    May 16, 2020 at 0:27
  • 1
    $\begingroup$ 4) when averaging between all items in the same batch of questions (for the same participant), you get a more continuous variable, and this helps for testing, both parametric and not. for instance, if someone puts 5, 5 and 4 to three items of some questionnaire, about the same issue, it results to be more convinced than someone else that puts three 5s as answers. 5) one can argue that the resulting average, coming from variables that are independent at least for some of their parts, is gaussian, and then t-test is appropriate. this is disputable, but t-tests are disputable almost always. $\endgroup$
    – carlo
    May 17, 2020 at 22:57
  • 1
    $\begingroup$ @carlo Ah! I see I see... I misunderstood your initial comment about means. Yes: sums of Likert items are going to be approximately normal, so game on. (My assumption was that America was considering a single Likert scale outcome.) $\endgroup$
    – Alexis
    May 18, 2020 at 2:57
  • 1
    $\begingroup$ I don't know about America, but I hope that when someone is doing t-tests on likert surveys, they have used multiple items averages. $\endgroup$
    – carlo
    May 18, 2020 at 7:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.