12 teachers are teaching 600 students. The 12 cohorts taught by these teachers range in size from 40 to 90 students, and we expect systematic differences between the cohorts, as graduate students were disproportionately allocated to particular cohorts, and previous experience has shown that the graduate students on average score considerably higher than the undergraduate students.
The teachers have graded all the papers in their cohort, and have assigned them a mark out of 100.
Each teacher has also looked at one randomly selected paper from three other teachers, and given it a mark out of 100. Each teacher has had three of his/her papers marked by another teacher. 36 different papers have thus been cross-marked in this way, and I call this my calibration data.
I can also see how many graduate students were in each cohort.
My questions are:
A) How can I use this calibration data to adjust the original marks in order to make them fairer? In particular, I'd like to wash out as much as possible the effects of overly generous/ungenerous makers.
B) How appropriate is my calibration data? I didn't have a choice in the rather limited 36 data points of calibration data I got in this course, and don't have any option to collect any more during the current semester. However, if this situation recurs I might be able to collect more calibration data or else collect different types of calibration data.
This question is a relative of a popular question I asked at: How can I best deal with the effects of markers with differing levels of generosity in grading student papers?. However, it's a different course and I'm not sure how useful reading that question would be as background for this current one, since the chief problem there was that I had no calibration data.