I am doing an analysis of paired survey responses. The responses are on a 5pt Likert scale and the same 100 respondents are asked a question at year 1 and year 2. I would like to do some pairwise comparisons to see if there was a significant change in one of the questions.

Example table:

$$ \begin{array}{lrrrr} \text{Cat} & \text{year}_1 & \text{year}_2 & & \\ \text{Strongly Agree} & 8 & 4 & & \\ \text{Agree} & 28 & 32 & & \\ \text{Neither } & 32 & 28 & & \\ \text{ Disagree} & 30 & 32 & & \\ \text{Strongly Disagree} & 2 & 4 & & \end{array} $$ In R

df <- data.frame(categories = c("strongly agree", "agree", "neither", 
"disagree", "strongly disagree"),
             year_1 = c(8, 28, 32, 30, 2), year_2 = c(4,32,28,32,4))

In order to preserve the categorical nature of the data I would like to not convert the data to numeric and then do a t-test.

The only thing I can think of is approaching this with a Bayesian model with a beta prior and a binomial likelihood with weak priors, then drawing from the posterior and comparing the posterior differences for the pairwise comparisons of each of the Likert scales. However, this does not correct for the correlation within the responses as the same people are responding to each question just at a different point in time.

Example code

a = 0.1; b = 0.1
mean(rbeta(10000, 4 + a, 100 - 4 +b) > rbeta(10000,8 + a, 100 - 4 +b))

I could do a mcnemar test but then I would be continuously dichotomising (and I really would rather not report only p-values. Id's rather have confidence intervals around the values) or Wilcoxon signed rank test. Ideally I want to establish confidence intervals around each response and see if they overlap.

Is there a better approach? Is the Bayesian approach the best for what I need? If so how do I adjust for the correlations for the individuals.

  • $\begingroup$ Despite what you say, I would analyze these data with a paired t-test. Likert data are considered ordinal rather than categorical. There are cumulative link mixed models, but they're less efficient, tough to fit and implement. More here $\endgroup$ – AdamO May 29 '18 at 15:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.