How to measure correlation between multi-item likert scales?

Question

I recently conducted a online survey and are now having trouble deciding for the right analysis method.

For example: I wanted to see if there is a correlation between Factor A and Factor B.

However in my survey I did not ask the participants to directly rate Factor A and Factor B, but rather asked multiple subquestions for these two factors which were rated from 5(Strongly Agree) to 1 (Strongly Disagree). I created a sum score for item A and a sum score for item B by summing up the values of each subquestion and calculated the mean of the sumscores to measure them. So my question ist: Which method would be best to use here? I want to test if e.g. people with a higher score in Factor A would be more likely to decide for Factor B. I actually planned using Spearman or ANOVA depending on the number of independent variables, but I am not sure if that would really be the correct way to measure the correlation.

Answer 1

Since you have a 5-level Likert scale, rather than encoding them as:

$$1, \, 2, \, 3, \, 4, \, 5$$

You could encode them as:

$$-2, -1, -0, \; \; \, 1, \; \; \, 2 \; \; \,$$

That way, summing over a large number of questions will give you a number that is positive if someone tends to agree and negative if someone tends to disagree. More importantly, you can interpret this number the same, regardless of the number of questions summed over.

Then you could simply plot the average response on factor A against the average response on factor B per participant in a scatter plot. This will give you much more compelling evidence then just some measure of correlation or a $p$-value from an analysis.

You could add a measure of correlation to the plot. If the relationship is more or less linear, you can use the Pearson correlation. Otherwise, you can use rank correlation instead, like Spearman's rank correlation.

The question only mentions factors A and B, so I see no reason to use ANOVA (comparing any number of means) over a simple correlation coefficient.

Answer 2

As you computed a sum score you accepted that this is data suitable for buidling quasi-metric data as in a Likert scale. The quasi-metric scale niveau gives you the option of computing either Pearson correlations or Spearman correlations.

people with a higher score in Factor A would be more likely to decide for Factor B

That is what Spearman correlations are made for. Pearson correlations are made for linear relationships. So choose Pearson if you expect a linear relationship and Spearman if you expect a "the more A, the more B" relationship indepent of linearity.

Should you have more then 2 scales and you want to investigate which of those "goes together" whith which other ones, you are probably looking for Factor analysis or Principal Componen Analysis rather then ANOVA.