I'm a bit stumped as to what test(s) is/are most appropriate for the following scenario. I found a couple of similar questions but wasn't sure if the answers applied.
For a report I am working on for my boss, I am trying to test whether there is a statistically significant difference (and find the effect size) between the official letter grades earned by students who took a course during Spring semester taught using a traditional lecture model and the grades of students who took the same course during Spring semester following a redesign (i.e., determine if performance in the course is independent of course design). I also want to determine for which individual letter grades there is a significant difference (i.e., proportion of A's pre versus post-redesign, etc.). For the moment, I am just using 1 IV and 1 DV. (I've not completely wrapped my head around ordinal logistic regression models and diagnostics in Stata yet) :)
A bit more about the data:
- Records in the data set represent individual students. I have the benefit of having access to the data for all students who have taken the course--not just a subset.
- The DV (course grade) takes on the values: "A", "B", "C", "D", "F", and "W" (withdrawal), coded from -1 ("W) to 4 ("A"). The variable is really more ordinal than truly interval, since there is not exactly the same "distance" between all grades (from A to C != D to W, for instance).
- The IV (course design) has two levels: before and after course redesign.
- For Spring semester, N = 1717 students who took the traditional course and N = 44 students who took it post-redesign.
- Spring grades are non-normally distributed, and the two groups have unequal variances.
- a cross-tabulation of grades and course design for spring semester that includes expected counts shows 3 of 12 cells with an expected frequency < 5, 1 cell with expected frequency < 1, so I know I'd be violating two of the assumptions of a Chi^2 Test of Independence
I compared Fall:Fall using a Chi^2 test for independence, calculating Kendall's tau-b as a measure of effect size (not sure if gamma would have been better?), and looking at the standardized residuals in each column of the crosstab (using
tabchi package) to determine significant difference for individual letter grades.
What are the most appropriate steps to take to analyze the Spring semester data?