How can I compare the results of two Likert scale surveys? For my thesis, I had to evaluate the usability of two desktop applications - A and B. 5 participants were asked to rate how much they agreed to a set of statements. A 5 point Likert scale was provided (Strongly disagree=1; Disagree=2; Neither agree nor disagree=3; Agree=4; Strongly Agree=5 ) to judge how much they agreed with each statement.
This questionnaire was first filled for application A and then for application B.
Now, I have to analyse the Likert scale data of both the samples using some sort of parametric or non-parametric test. But I am a bit confused on how I should proceed and everything I have read online has not been of much help.
I understant I can conduct a t-test but I do not know how my data should look like? Do I have to feed in the data based on the mean results of each statement for each of the applications?
I just want to compare the results and show that 'X' application is ranked to be much more useable.
 A: So McElreath (2020) talks about how Likert data is often the most mistreated type of data. While other methods may give you a reasonable answer, what you really should be using is an ordinal logistic regression. Here is a brief introduction that might be useful:
https://stats.idre.ucla.edu/r/dae/ordinal-logistic-regression/
but I'd recommend checking out an ebook copy of McElreath's book from your university library (they should have it). You can also watch this lecture on this type of data from his graduate seminar:
https://www.youtube.com/watch?v=zA3Jxv8LOrA&list=PLDcUM9US4XdNM4Edgs7weiyIguLSToZRI&index=14
McElreath, R. (2020). Statistical rethinking: A Bayesian course with examples in R and Stan. CRC press.
EDIT: Adding another citation here that will be relevant to those seeing this:
Bürkner, P. C., & Vuorre, M. (2019). Ordinal regression models in psychology: A tutorial. Advances in Methods and Practices in Psychological Science, 2(1), 77-101.
A: Given that you have used a likert scale, comparig means is a plausible way of seeing if the two apps are different, but because you only have 5 people in each group, you shouldn't use the t test because you don't have a large enough sample to assume that the mean will be normally distributed.
If the participants rating both apps then your best bet is a Wilcoxon signed-rank test which is like a paired t test but with looser assumptions.
If you had wanted to do a ranking, then you could have told the participants to rank the two, forcing them to choose which one they prefer. You can then simply compare that to a 50/50 even split which would have been what would have happened had they chosen randomly.
Some comments if I may: With 15 items, chances are you're going to get something that falsely appears significant. I genuinely think you've got to select only like 1 item that is the best comparison  between the two apps and run the test on that. There are non parametric multivariate tests (i.e. they let you compare lots of means all at once, while having only a small sample), but it gets pretty complicated. You might not even need to hypothesis test honestly, why not just use some summary statistics.
