0
$\begingroup$

I have the results from two surveys taken approximately one year apart. All of the questions are Likert-type; many questions are the same between the two surveys (these are the ones in which I am interested). Some of the survey respondents may or may not be the same (the results are anonymous). For the first survey, I have some 180 responses; for the second, 71.

What is the best way to compare responses to the questions that are repeated between surveys? I'm really looking to answer questions like this: Question (number) had 14% of the respondents score 1 in Year A (i.e., 25 out of 180), but 20% (14 out of 71) in Year B. Is there a real difference between those two scores, or is this difference not statistically significant?

Improve this question
$\endgroup$
0

2 Answers 2

Reset to default
1
$\begingroup$

Hierarchical ordered logistic regression. Example of implementation in Stata https://www.stata.com/features/overview/multilevel-ordered-logistic-models/

Improve this answer
$\endgroup$
3
  • $\begingroup$ Excellent - I assume there are ways to do this in R? Do you happen to have any references handy? $\endgroup$
    – Karl Wolfschtagg
    Oct 6, 2022 at 18:55
  • $\begingroup$ stats.stackexchange.com/questions/238581/… $\endgroup$
    – socialscientist
    Oct 6, 2022 at 21:12
  • $\begingroup$ Big picture is that you model an overall response to your survey item and time-soecific deviations from the mean (i.e. the differences from that mean for the 2 time periods of your data). If you put your data in a long format where each row is a response to a at time T and fit the model to the question asked at 2 points in time, you can do something like lmer(response ~ (1| time)). To accommodate the ordinal response, you'll want the cumulative logistic or probit distribution. If this is over your head, you can always just fit a linear regression to that data with a dummy variable for time. $\endgroup$
    – socialscientist
    Oct 6, 2022 at 21:15
1
$\begingroup$

As socialscientist notes, you could use some sort of hierarchial/multilevel ordinal regression model (e.g., see Bürkner & Vouree, 2019 for more information). Though hierarchical models can be somewhat complex, and you may not be familiar with them. So you could also try estimating the polychoric correlation for each item given to both sets of survey takers. The polychoric correlation is a correlation commonly used with ordinal data, and there are many good articles on the topic (e.g., Uebersax, 2015).

Regarding R packages, see brms for ordinal regression, and polychor for polychoric correlations.

References

Bürkner, P. C., & Vuorre, M. (2019). Ordinal regression models in psychology: A tutorial. Advances in Methods and Practices in Psychological Science, 2(1), 77-101.

Uebersax, J. S. (2015). Introduction to the tetrachoric and polychoric correlation coefficients. URL: http://www. john-uebersax. com/stat/tetra. htm (visited on 12/18/2014).

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.