I'm looking at the exam results of a university course. The exam was completed by 500 people, who each completed 105 multiple choice questions. I have each person's total score on the exam (ExamTotalScore).

11 of those questions were on a topic that was quite unrelated to the other questions within the course, and I can separate out scores on that part of the exam (ExamTopic1Score). The mean ExamTopic1Score is 7.5, SD 2.4, range 0 to 11.

Students were required to complete 54 training questions on Topic 1 prior to the exam. I can see their score out of 54 (Topic1TrainingScore).

I'm interested to assess the efficacy of the training materials for Topic 1. Since it's impractical for me to do that in an experimental way, I was thinking of constructing a simple model in which Topic1TrainingScore predicts ExamTopic1Score.

One objection to that model is that people who are skilled at exams are generally conscientious and would tend to do better on the training materials even if they were useless. As some measure of control for that I thought of including a covariate.

Which of these covariates is more appropriate, and why?

  1. Use ExamTotalScore as a covariate.
  2. Make a new variable ExamNonTopic1TotalScore and use that as covariate. That is, ExamTotalScore after scores on the Topic1 questions have been subtracted out.
  3. Use something else as a covariate.
  4. Don't use a covariate.

My initial inclination was with '2', since otherwise the outcome variable ExamTopic1Score and the covariate ExamTotalScore are very linked, in that an extra point for a student on ExamTopic1Score automatically means an extra point ExamTotalScore. However, I'm not sure how to explain why this is a problem.

Also, I would welcome general impressions on whether my approach is appropriate.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.