I have the following contingency table showing student grades in 2017 and 2018. Student populations are different in both years. In 2018, a different exam format is introduced. The research questions are (1) is there a significant difference in the grades of students between the two years, and (2) if there is a difference, in which of the examinations students did better.

Grade   2017    2018
A       75      60
B       176     110
C       85      84
F       81      56

I have looked at this and this but being not stats-savvy, I would like some pointers to (possibly newbie-friendly) answers to the two research questions.


It seems you have already seen a similar analysis using R. Here is output from Minitab 17. Observed cell counts are consistently about the same as expected cell counts and standardized residuals are all small in absolute value. There is no reason to suspect grade distributions differ between the two years.

Rows:    Columns: Worksheet columns

          2017     2018  All

A           75       60  135
         77.43    57.57
       -0.2767   0.3209

B          176      110  286
        164.05   121.95
        0.9333  -1.0824

C           85       84  169
         96.94    72.06
       -1.2124   1.4061

F           81       56  137
         78.58    58.42
        0.2728  -0.3164

All        417      310  727

Cell Contents:      Count
                    Expected count
                    Standardized residual

Pearson Chi-Square = 5.844, DF = 3, P-Value = 0.119
Likelihood Ratio Chi-Square = 5.827, DF = 3, P-Value = 0.120

Of course the proportional distributions of grades A, B, C, F are not exactly the same in the two years, but the chi-squared test shows no evidence of statistically significant differences.

In view of this, it hardly seems worthwhile to try to answer whether grades in one year were 'better' than in the other. However, it may be worthwhile to consider how this could be done if a difference were suspected. One method would be to assign numerical values to the letter grades, perhaps A=4, B=3, C=2, F=0.

If the original data on all the students are available, then it should be easy to substitute numbers for grades. But using R, it is relatively easy to construct numerical vectors x.17 and x.18:

x.17 = rep(c(4, 3, 2, 0), times=c(75, 176,85,81))
  0   2   3   4 
 81  85 176  75                         # verify same as in contingency table
summary(x.17); sd(x.17); length(x.17)
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
  0.000   2.000   3.000   2.393   3.000   4.000 
[1] 1.329674                            # sample SD 2017
[1] 417                                 # sample size 2018

x.18 = rep(c(4, 3, 2, 0), times=c(60, 110,30,56))
  0   2   3   4 
 56  30 110  60 
summary(x.18); sd(x.18); length(x.18)
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
  0.000   2.000   3.000   2.461   3.000   4.000 
[1] 1.427475
[1] 256

The sample lower quartiles, medians, and upper quartiles are the same for the two years. The sample means are very similar. It is not surprising that a Welch (separate-variances) two-sample t test does not show a significant difference in these numerical scores. [Data are not normal, but sample sizes are moderately large, and there are no outliers or indications of extreme skewness, so it seems reasonable to use a t test, known to be robust in these circumstances. In view of the fact that the data take so few distinct values, I would be reluctant to use a Wilcoxon rank sum test.]

t.test(x.17, x.18)

        Welch Two Sample t-test

data:  x.17 and x.18
t = -0.6125, df = 510.27, p-value = 0.5405
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -0.2846483  0.1493440
sample estimates:
mean of x mean of y 
 2.393285  2.460938 

In summary, there is no reason to believe student performance was significantly different in in one year than in the other.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.