2
$\begingroup$

I have data from ~3500 students regarding their experiences taking an online class last year. The class consisted of a unified on-demand component and several sessions on Zoom with instructors. I didn't design the survey, but I've been given the results in a spreadsheet to analyze.

The data looks like this (each row represents a student's response):

Instructor Name I found the software easy to use (1-5 Likert scale response) More Likert scale questions... Did you use a smartphone or tablet for the course (answer can be 1 or 0) Did you use a laptop? Did you use a desktop? More Likert scale questions...(This time about the live classes) What did you find helpful about the course (long answer questions which I've coded into five categories) What could be improved about the course? (Similarly, I've coded these into five broad categories)
Alex Alehead 3 2 1 1 0 4 I liked discussing the issues in breakout rooms. (Coded as 1 - online discussions) I felt the instructor could have more clearly outlined the syllabus (Coded as 3 - instructor communication)

I have no real experience with statistics, but it seems to me like there are things that could be discovered from this data apart from the averages and counts.

Is there a way, for example, to account for the differences between instructors when looking at course satisfaction? (My instinct tells me that a poor instructor will drag all the numbers down)

Another example: is there a way I can see if the type of, or the number of devices used affects the student's perception of the software used in the course?

Sorry for the barrage of questions! My main question is: How would you approach this data set? What tests would you run on it?

$\endgroup$
4
  • $\begingroup$ A variety of nonparametric tests (e.g, Likert scale differences among instructors) and chi-squared tests for devices vs perception of software. Your question is not readily answerable as is. Too many issues too few comments on specific objectives.// This site works best with one targeted question at a time. // Please give a better explanation of what Likert scores are available. Your 'table' hard to read and not very informative. // Will give brief answer about instructors vs Likert satisfaction scores. $\endgroup$ – BruceET Jun 5 at 0:33
  • $\begingroup$ Bruce, thank you very much for the feedback! I'm sorry the question is poorly presented. The main purpose of the analysis is for me to present to the faculty what worked well, what went poorly, and what suggestions could be made for further changes to the course. $\endgroup$ – Joshua Rapp Jun 5 at 1:19
  • $\begingroup$ If there's a correlation between particular device usage and satisfaction, for example, that's something that we might be able to explore when making suggestions to the students. $\endgroup$ – Joshua Rapp Jun 5 at 1:21
  • $\begingroup$ Other Likert Scale questions include 'I am satisfied with the on demand classes' (Strongly disagree to strongly agree), 'I am satisfied with the live classes', 'I feel the course was well organized and planned', 'I feel the course was effective in achieving my learning goals' $\endgroup$ – Joshua Rapp Jun 5 at 1:24
1
$\begingroup$

Suppose there are four instructors and one question about overall course satisfaction.

Then Likert scores for the four instructors might be something like the fictitious data below. (I'm using 1000 students; 250 per instructor.)

set.seed(2021)
x1 = sample(1:5, 250, rep=T, p = c(1,2,3,2,1))
x2 = sample(1:5, 250, rep=T, p = c(1,1,2,3,2))
x3 = sample(1:5, 250, rep=T, p = c(1,2,2,2,3))
x4 = sample(1:5, 250, rep=T, p = c(2,2,1,3,2))
x = c(x1,x2,x3,x4)
g = rep(1:4, each=250)

With only five different values, boxplots do not show much detail, but it is clear that not all five instructors have the same median score (vertical bars inside boxes). Nonoverlapping 'notches' in the sides of boxes are nonparametric confidence intervals for medians. suggest differences between two instructors. [For Instructor 2, the notch extends beyond the high edge of the box.]

boxplot(x ~ g, col="skyblue2", horizontal=T)

enter image description here

A Kruskal-Wallis test shows significant differences among instructor's scores with a P-value near $0.$

kruskal.test(x~g)

        Kruskal-Wallis rank sum test

data:  x by g
Kruskal-Wallis chi-squared = 39.018, df = 3, p-value = 1.721e-08

The notches in boxplots suggest where the most important differences may lie. One method (not necessarily the best) to do formal tests is to look at two-sample Wilcoxon rank sum tests.

For example, such a test confirms that there are significant differences between Instructors 3 and 4.

        Wilcoxon rank sum test with continuity correction

data:  x3 and x4
W = 36920, p-value = 0.0003243
alternative hypothesis: true location shift is not equal to 0

One has to be careful to avoid 'false discovery' upon repeated testing on the same data, but with such a small P-value this comparison is solid.

This brief answer illustrates one possible avenue for analyzing your data. It is enough of my suggesting possibilities in terms of fictitious data. After some thought, you may want to present specific data and questions about specific analyses.

$\endgroup$
1
  • 1
    $\begingroup$ Thanks Bruce, this is helpful. I don't have any experience dealing with statistics (I'm a computer scientist who has been given the task), but this looks like a good place to start! $\endgroup$ – Joshua Rapp Jun 5 at 1:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.