0
$\begingroup$

I have some questionnaire responses that are in the form of a likert scale (1 = Excellent, 2 = good, 3 = unsure, 2 = not good, 5 = Bad etc.). I have 12 different questions in this format and a respondent number of 81.

I have many factors that I want to check for significance with regards to the above 12 outcomes. For example: profession, workplace, research area, as well as some binary yes/no responses.

One example is: is there a difference between PhD students' and Professors' opinions with regards to animal research (1-5 likert response).

Is a Logistic Regression the right way to go about analysing this data? Because I have so many different outcomes I want to test, I was not sure if this would generate too many P-values and thus create too much room for error.

Thank you.

$\endgroup$
3
  • $\begingroup$ Parenthetically, you probably shouldn't use "unsure" for a neutral response in a Likert item. $\endgroup$ Commented Jul 22, 2017 at 17:46
  • $\begingroup$ Okay - I will look into that further, thanks. I thought I had to give an option for having no feeling / response either way. $\endgroup$
    – user170830
    Commented Jul 22, 2017 at 17:49
  • $\begingroup$ To me, "unsure" doesn't mean "neutral". Neutral" means "I know that I think the answer should be about 3 on a scale of 1 to 5.". "Don't know" or "unsure" would be separate opt-out answers. $\endgroup$ Commented Jul 22, 2017 at 18:11

2 Answers 2

1
$\begingroup$

Results from a single Likert item should be considered ordinal in nature.

If you want to do a simple test like compare responses between two groups, you can use the Cochran-Armitage test. Some permutation tests can also be employed with ordinal response variables. Some people think it's fine to use the Mann-Whitney in these cases, and others disagree.

For anything more complicated, ordinal regression can be used.

$\endgroup$
3
  • $\begingroup$ Thank you so much. I see you are correct and that I must use Ordinal Regression. Can an Ordinal Regression test take more than one independent variable? I want to test a Likert scale response (1-5) among five different groups of people to look for differences among them, and I am not sure if we have the same requirements for binary data as with Logistic Regression. Finally, there are some differences among my sample sizes. Group 1 has 10 less respondents than group 2, which has a further 5 respondents less than group 3 (n=less than 50 for all so I think this is important?). Thanks so much. $\endgroup$
    – user170830
    Commented Jul 24, 2017 at 8:12
  • $\begingroup$ Depending on your software package, ordinal regression should be as flexible as ordinary least squares general linear model. That is, you should be able to have multiple IV's, categorical or continuous IV's, interactions, etc. I don't think the sample sizes present a theoretical difficulty with the regression, but I would check the documentation on the specific software you use. Five groups where the smallest n is 7 (???) might give the algorithm some difficultly. Not sure.... But also, if you are just comparing among five groups without other factors, you might look to (cont) $\endgroup$ Commented Jul 24, 2017 at 11:00
  • $\begingroup$ (cont) an extended Cochran-Armitage test that compares among more than two groups, or Kruskal-Wallis with Dunn test (with the caveat that some people don't like using this test with Likert item data). $\endgroup$ Commented Jul 24, 2017 at 11:02
0
$\begingroup$

Different field seem to take a different approach to whether responses to a Likert-type scale can be submitted to parametric or non-parametric analyses. Technically, they are ordinal in nature, though there have been studies of scale use demonstrating that people's responses often reflect a quasi-interval scale (though there's no guarantee). Pragmatically, check out what the standard is in your particular field, but verify that any parametric results you might obtain (e.g., based on t-test, ANOVA, OLS regression) are confirmed by non-parametric tests, such as the one's mentioned by Sal Mangiafico.

The fact that your posing the question concerning the use of logistic regression is slightly puzzling because that particular analytical tool typically refers to the analysis of binary outcomes. While you mention binary outcomes, the majority of your post seems to refer to continuous (Likert-type) survey responses. Logistic regression only applies to binary outcomes.

Yet, the bigger question you pose concerns the large number of significance tests you plan to conduct, and you seem to be concerned about an inflation of the alpha error. Depending on how deep you want to get into this topic, this question can get tricky very quickly as any need for adjustment of the alpha criterion depends on whether you choose to rely on "experiment-wise" or "family-wise" alpha errors. Again, my recommendation would be to look at what's customary in your particular field. Often, you will see that researchers do not engage in any kind of adjustment of the alpha error when reporting survey results such as yours. However, given that your concern is a good one, in the parametric world of things one classical way of tackling this particular issue has been through the use of MANOVA-based "step-down" procedure. To the extent that you have 2+ parametric outcome variables, which are also correlated with one another, you submit all of them at the same time to a MANOVA in which you use a set of continuous or categorical predictor that should apply to all outcomes. You will obtain a multivariate effect for each predictor, which tells you, whether across the string of outcome variables (a vector) this particular predictor is related to variation. If you do observe a significant multivariate effect, you still don't know for which of the specific outcomes (of the set of 2+ outcomes) the predictor has a significant effect. Thus, in the presence of a significant multivariate effect you examine the univariate effects, and have "license" to interpret them, if significant. (The implication of this method is that you should not interpret any significant univariate effect in the absence of a non-significant multivariate effect.) The focus on the multivariate effects helps protect against redundant univariate effects, i.e. you observe that X has an effect on Y1, but also on X on Y2, even though Y1 and Y2 might be highly correlated.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.