1
$\begingroup$

Can we test statistically that 64 students of 249 examinees of a particular school each naturally scored exactly 95 marks in a national exam taken by approx. 300,000 students each year without "cheating" ? If so, what are the confidence levels / certainities.

The exam marks can range between 0 and 100. It is a subjective exam having 28 questions with marks ranging from 1 mark to 6 marks per question and the detailed marking scheme is specific to a 1/2 mark and strictly followed. The national results of this exam in past years had a somewhat bi-normal (2 mounds shaped) mark distribution like in this image

mark distribution

A complaint of mass cheating at this school is reported to us and we want to know if statistical methods also independently indicates the possibility. We are also unsure if national statistics can be applied to a specific group of students and if there is prior research on this.

EDIT (in response to whuber)

We are noobs at this, and estimated the probability of a random student getting exactly 95 marks as 7 out of 1000 (0.007) for this exam. Then 249Comb64*(0.007)^64 gave us astonishing odds and then adding the cases for 96..100 didn't help much improving the odds. As mentioned in the comments the exam papers were leaked 2 days in advance and not cancelled / retested, and hundreds of schools nationwide are throwing up such odd results.

EDIT (in response to whuber)

Here is the mark distribution image for all students (2,067) of 13 schools in the vicinty for the same paper (363 got exactly 95). The question paper and an unofficial markscheme (with 1 significant error for 4 marks) were circulating 2 days before the exam and had been faxed to the head of the examing body 3 days before the exam by a whistleblower. But the board did not cancel the exam and insists all is well with their exam and the marks will stand.

https://twitter.com/kanchangupta/status/979420618788491265

mark distribution

$\endgroup$
  • 1
    $\begingroup$ See en.wikipedia.org/wiki/Stand_and_Deliver#Plot for a real example. $\endgroup$ – whuber Jul 27 '18 at 13:32
  • $\begingroup$ Many thanks for the link. I had seen the film. However, the exams in question have been under heavy public criticism this year for mass paper leakages upto 2 days before the exam. The exam board did not cancel or retest this particular paper, and similar (or even more "abnormal") results are being reported in hundreds of schools across the country. $\endgroup$ – user215877 Jul 27 '18 at 14:09
  • 3
    $\begingroup$ That's useful information and suggests performing a Bayesian analysis of the results--but it does not justify computing probabilities as if exam responses are independent and random. That's the whole point of the movie: the responses of students in one class were not independent. $\endgroup$ – whuber Jul 27 '18 at 14:44
  • 3
    $\begingroup$ Even without any cause for suspicion, one could not reasonably treat the students from the same school as independent, so any probability calculation that made sense would need some model for how their answers would be dependent (and this is exactly where whuber's link would come in as a clear pointer than one cannot simply handwave away such considerations). How, then, to figure out whether the marks are "too alike" compared to students who merely studied together, learned off the same teacher and off each other? (Do you have good data on those students in similar situations?) $\endgroup$ – Glen_b -Reinstate Monica Jul 28 '18 at 3:10
  • 1
    $\begingroup$ Here is a link in whjich an Australian statistician said of this spike / distribution "these scores bear no resemblance to any known distribution, and defy explanation.”. priceonomics.com/a-fishy-looking-test-score-distribution $\endgroup$ – user215877 Jul 28 '18 at 14:27
1
$\begingroup$

I am very intimidated by the lack of responses to this question; but i'll give it a stab. I am just a student, so please don't pile on the response. Maybe my response is incredibly naive but at least it's something.

I would take a sufficiently large sample of similar schools (eg.30 schools) and record how many students got exactly 95 (or simply very high scores). divide that by the total number of students in each school to get a proportion. If you assume there is no cohort effect where the school in question improves over time, you can simply go through the old test results from this very same school.

Then you can simply do a T test against the claimed value of what proportion got that score in the current year/school. i.e. (sample mean of proportions - claimed value)/(sample standard deviation/number of schools).

Then look up a T test p values chart, with degrees of freedom=number of schools minus 1. If the p value is less than 0.05, it's sufficient to say this is abnormally high--the lower the p value the more significant the gap between what you'd normally expect by chance and what you got at the current school.

If the sample is sufficiently large then central limit theorem kicks in and it doesn't matter what the original distribution is; the test is on the means, it'll work anyway. 30 is a rough guide, 13 schools might be enough, though try to get more. It won't matter that students are not independent of each other; we're averaging between schools not the students, as long as the schools are independent you should be fine. You can't tell which student cheated, but you can tell that this amount of 95's is abnormally high compared to other similar schools.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.