How to: Analysing 5-Point Likert Scale with ANOVA

Question

I am very confused since I could not find an understandable answer for it: I must analyse my Likert Scale data with ANOVA to get the significance (using the ANOVA is the requirement...).

I have to evaluate $6$ Emotions performed by two different faces (stilized face and real face). Each Emotion has $4$ Question Categories which contain $4$ questions each. These questions use a $5$ point likert scale in which the subjects rate how pleasant the face appears to them.

What I want to do is compare each question of the stilized faces (ist that even neccessary? Or would a comparison of the question categories suffice?) to the same question of the real faces, to find out if there is a significant difference concerning the rating. Then do the same with the question categories (I don't know how I could do that then)

My biggest issue is that I cannot figure out how to approach this. What do I do with my Table full of likert scales in order to be able to compare the questions? Do I just build the overall sum of the whole question, or do I have to multiply the likert-values of each cell with a number fitting for that likert scale? (example: $5$ is the best rating so I will multiply the amount of $5$s in each row with $+2$, the amount of $4$s with $+1$, $3$s with $0, 2$s with $-1$ and $1$s with $-2$?).

To compare them I would assume my $H_0$ to be : $H_0$ : There is no difference in the rating of how pleasant a subject perceives a stylized face or real face.

Answer 1

Ok, so Likert scales are often used in psychology as continuous variables. From your question, it sounds like this is the case.

What comes after really depends a LOT on what your question asks. Do you want to compare question categories in different conditions (stylized vs not) or compare emotions in different conditions (stylized vs not). Can the question categories be combined or not. This affects the way you are combining the groups together in order to calculate your sums of squares.

You say that you want to compare the means of the individual questions across conditions. In which case a student T test will do. But i'm guessing the question expects more.

So, if you want to calculate question categories across conditions (again, be sure that this is what you want to do), then here is a rough guide.

First calculate your grand mean for each question category across both the conditions. Now take your individual values and subtract them from the grand mean. Now square it. Now add those results together. That is your total sums of squares.

Next, you'll want to calculate your treatment sums of squares (sometimes called between sums of squares--i.e. between treatments). Average all the values in one question category of one condition. (adding the means of each question and dividing by how many questions is the same as adding all the individual values and dividing by the number of people, you can confirm this for yourself). Subtract your grand mean from it. Square it. Times it by how many people who have answered that question category. Repeat it for your other condition. Now add them.

Now take your total sums of squares and subtract your treatment sums of squares. This is your Sums of squares error (sometimes called sums of squares within)

With these three values you are ready to do an F test. Take your sum of squares treatment and divide it by (the treatment degrees of freedom) how many groups you have minus one. Now this goes on top of a fraction.

Now work out the bottom of the fraction. Take your sums of squares error and divide it by (the error degrees of freedom) how many groups you have times one less than how many people you have (m*(n-1)).

Work out the fraction.

Now with this number, go look up an F table at 0.05, where the first degree of freedom is the treatment degrees of freedom, and the second degree of freedom is the error one. If your number is bigger than that then you have significance.

Your null hypothesis from what you've written is there is no difference in mean of question categories between conditions. You reject that, and say there is a difference.

This allows you to say that the question category is different across conditions. You don't know exactly which question set it off, but you might not want to know that, you just want to be able to say that, overall, there's a difference in there somewhere. You might want to planned contrasts to see where exactly that difference is. i.e. t tests. (this is why i think ANOVAs are useless because you always end up doing the contrasts anyway, so why not just do them from the start, but that's another story, courses always teach ANOVA and you will always have to learn ANOVA)

Again, I wish to stress I am only going of what you've written, it is entirely possible to group them up in different ways. e.g. emotions across conditions sounds very plausible.

Either way, Khan academy should really help you out, here he literally calculates an F statistic the way I have described by hand just to show you.

Answer 2

Thanks a lot! While I was waiting for an answer I figured out a way to do it which could be equal to what you described (almost) though automated since excel has an function for ANOVA.

What I did was: I calculated the mean of every question within every Question Categorie but before I multiplied the amount of likert-skala choices for one value with their respective number ("completely true"-Amount * 5, ... "Neutral" * 3, ... "Not true at all" * 1,

which left me with a table (just a random example I wrote down which in real-life is a column not a row) like this: Emotion 1 Question Group 1 - Stilized: 5,3,4,1 Real: 4,3,3,2 On these two I was able to use ANOVA. I have one categorie of which I know that all questions in it are significantly different which I use as a way to check if I did it right at all and it seemed to be right(-ish)? (all were significantly different)

Now If I understand what you said correctly it is obsolete (is it wrong?) if I multiply my Likert-Values with their according variable-values (as I did). All I need to do is what I did minus the multiplication?

Thanks in advance and thanks again for the quick answer!

Answer 3

If by Likert-skala you mean Likert scale then i think so. (I'm not trying to be condescending, i'm genuinely trying to be careful in my answer).

Likert scales let you rate things and they are normally treated as just numbers. eg. never 0 rarely 1 sometimes 2 often 4 always 5. Maybe, from what you're describing, your data has been left with just the words in which case just change them to the corresponding numbers. Then they can just be summed up like normal. Or maybe you already have the number, then just leave them alone (if I am reading you correctly).

Sometimes Likert scales can be treated as ordinal. But to do an ANOVA then yeah they'll have to be treated as scales.

If your teacher wants you to rescale them in some way, for example where "completely true" is 2*5 for example, then they're going to have to tell you that specifically. I'm guessing they don't though because that's really weird.

PS if you find my responses helpful then please rate them up thanks.

Answer 4

A few comments:

• It doesn't matter if you code the responses as {1,2,3,4,5} or {-2,-1,0,1,2}. Obviously the numerical result will be different, but the analysis comes out the same. It just depends how you would like to present the results.
• Similarly, it doesn't matter if the result for each Question Category is calculated from the average of the responses from the individual questions or the sum of the responses. Unless you have missing responses. In that case you will have to think carefully about how you will handle missing data.
• Usually if your questionnaire is designed to have questions summed into Question Categories --- that is, if individual Likert items are summed into Likert scales ---, then you wouldn't address the individual questions.