Analysis Course Survey Results

Question

I have data from ~3500 students regarding their experiences taking an online class last year. The class consisted of a unified on-demand component and several sessions on Zoom with instructors. I didn't design the survey, but I've been given the results in a spreadsheet to analyze.

The data looks like this (each row represents a student's response):

Instructor Name	I found the software easy to use (1-5 Likert scale response)	More Likert scale questions...	Did you use a smartphone or tablet for the course (answer can be 1 or 0)	Did you use a laptop?	Did you use a desktop?	More Likert scale questions...(This time about the live classes)	What did you find helpful about the course (long answer questions which I've coded into five categories)	What could be improved about the course? (Similarly, I've coded these into five broad categories)
Alex Alehead	3	2	1	1	0	4	I liked discussing the issues in breakout rooms. (Coded as 1 - online discussions)	I felt the instructor could have more clearly outlined the syllabus (Coded as 3 - instructor communication)

I have no real experience with statistics, but it seems to me like there are things that could be discovered from this data apart from the averages and counts.

Is there a way, for example, to account for the differences between instructors when looking at course satisfaction? (My instinct tells me that a poor instructor will drag all the numbers down)

Another example: is there a way I can see if the type of, or the number of devices used affects the student's perception of the software used in the course?

Sorry for the barrage of questions! My main question is: How would you approach this data set? What tests would you run on it?

Answer 1

Suppose there are four instructors and one question about overall course satisfaction.

Then Likert scores for the four instructors might be something like the fictitious data below. (I'm using 1000 students; 250 per instructor.)

set.seed(2021)
x1 = sample(1:5, 250, rep=T, p = c(1,2,3,2,1))
x2 = sample(1:5, 250, rep=T, p = c(1,1,2,3,2))
x3 = sample(1:5, 250, rep=T, p = c(1,2,2,2,3))
x4 = sample(1:5, 250, rep=T, p = c(2,2,1,3,2))
x = c(x1,x2,x3,x4)
g = rep(1:4, each=250)

With only five different values, boxplots do not show much detail, but it is clear that not all five instructors have the same median score (vertical bars inside boxes). Nonoverlapping 'notches' in the sides of boxes are nonparametric confidence intervals for medians. suggest differences between two instructors. [For Instructor 2, the notch extends beyond the high edge of the box.]

boxplot(x ~ g, col="skyblue2", horizontal=T)

A Kruskal-Wallis test shows significant differences among instructor's scores with a P-value near $0.$

kruskal.test(x~g)

        Kruskal-Wallis rank sum test

data:  x by g
Kruskal-Wallis chi-squared = 39.018, df = 3, p-value = 1.721e-08

The notches in boxplots suggest where the most important differences may lie. One method (not necessarily the best) to do formal tests is to look at two-sample Wilcoxon rank sum tests.

For example, such a test confirms that there are significant differences between Instructors 3 and 4.

        Wilcoxon rank sum test with continuity correction

data:  x3 and x4
W = 36920, p-value = 0.0003243
alternative hypothesis: true location shift is not equal to 0

One has to be careful to avoid 'false discovery' upon repeated testing on the same data, but with such a small P-value this comparison is solid.

This brief answer illustrates one possible avenue for analyzing your data. It is enough of my suggesting possibilities in terms of fictitious data. After some thought, you may want to present specific data and questions about specific analyses.