3
$\begingroup$

Are there any guide lines on determining which test to use?

For example, given 100 subjects with both Exam A and Exam B, and some student observations for those exams, i want to compare which exam is harder.

More specifically, I have 100 different subjects (ex. math, french, english, ...etc), and each subjects consists 2 exams, exam A & exam B. Now, I distribute these exams to a class of 100 and if a student score above 50%, then the observation is set as 1 and 0 otherwise.

An example table:

Subject Exam    Class Score
 MATH    A           90
 MATH    B           10
FRENCH   A           51
FRENCH   B           49
...      ...        ...

Which statistical test should i perform if I want to compare which exam is harder?

Summary of the question (by @user2974951):

  • We have 100 students, each student takes test A and B for 100 different subjects. The results of the tests are independent, so the results of test A does not affect the result of test B, and a student score in subject 1 will have no effect on the score of subject 2. The data is already aggregated on the subject and exam level, that is we only have the frequency of students which passed. Our goal is to determine whether test A is harder than test B, based on the aggregated data.
Improve this question
$\endgroup$
9
  • $\begingroup$ Don't do that, don't classify the scores into classes. Use raw scores to compute means rather. Are any of the observations dependent? That is, does a student take more than one test? Is a student present in more than one subject? Does a student take both the A and B test? $\endgroup$
    – user2974951
    Feb 25, 2021 at 11:44
  • $\begingroup$ @user2974951 unfortunately I am only given this type of data.. the observations are independent. It is true that all 100 students will take both tests for all subjects, but we assume that taking exam A will have no impact when taking exam B, and no relationship between each students. $\endgroup$
    – Math Avengers
    Feb 25, 2021 at 11:54
  • $\begingroup$ If possible, I would analyze data by student, not by class. I would use a multilevel model where student is level 1, class is level 2. This would allow me to assess the degree to which the class predicts scores (i.e. teacher effect) versus student. In R, this could be something like: fit <- lmer(score ~ exam + subject + (exam | class)). See benwhalley.github.io/just-enough-r/fitting-models.html, for examples $\endgroup$
    – Brant Inman
    Feb 25, 2021 at 15:02
  • $\begingroup$ @BrantInman But i want to compare exam A & exam B tho $\endgroup$
    – Math Avengers
    Feb 26, 2021 at 1:09
  • $\begingroup$ I think i should write the question clearer; the column "class score" is actually mentioning the same class with the same 100 students. So there's only one class level and students would be 100 levels $\endgroup$
    – Math Avengers
    Feb 26, 2021 at 1:43

1 Answer 1

Reset to default
0
$\begingroup$

Given the assumption that all observations are independent, that is within the subjects and between the subjects, then a simple t-test can be used to solve this - assuming all the assumptions of the t-test are valid (your data is not really continuous so they might not hold, but it might be a good approximation). Otherwise an equivalent non-parametric test might work better.

Your goal is to determine whether test A is harder than test B, or to put it differently if the frequency of test A is higher than that of test B. Given that you only have aggregated data, the results are interpreted on the frequencies of students which passed, rather than say the average score.

So for ex. you might find that the expected frequency for test A is 67, while the expected frequency for test B is 54. If the standard error is small enough you would find that the difference is statistically significant.

If, however, there is a relation between the exams, then you can use a paired t-test. If there is a relation within the students, then a linear mixed model can be used.

Improve this answer
$\endgroup$
2
  • $\begingroup$ For the example (...the expected frequency for test A is 67, while the expected frequency for test B is 54...) , what do you mean by the standard error? Of the differences between test A and B, or within test A/B? $\endgroup$
    – Math Avengers
    Mar 3, 2021 at 6:32
  • 1
    $\begingroup$ @MathAvengers A t-test would calculate the pooled standard deviation of the means (standard error), assuming equal variances. And it would use this standard error, combined with the mean difference between A and B to construct a confidence interval, and determine whether the result is statistically significant. $\endgroup$
    – user2974951
    Mar 3, 2021 at 7:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.