7
$\begingroup$

A previous year's grading curve for an exam I will be sitting is : enter image description here

Assuming the exam has 100 (or any number of) questions each equally weighted, will it take fewer extra correct answers to move up percentiles as you move away from the average?

E.g. will I need fewer extra correct answers to move from the 60th percentile to the 70th percentile, than from 45th to 55th percentile? And if so, why?

My reason for believing this is the case is found in another other curve that I have from a similarly marked exam:

enter image description here

This chart doesn't show percentiles, but from looking at it it seems it requires less extra marks to go up (or down) percentiles as you move away from the average. Note: 1 asterisk represents 4 students. The numbers on the LHS indicate how many raw questions correct.*

However, I can't think up a formal reason for this (my statistics knowledge is limited).

Or is it the case that say 1 extra mark on the exam would move me up the percentile curve the same amount regardless of where on the curve I am originally sitting?

FYI : This exam doesn't give us an A B or C depending on what % out of 100 we get. It simply ranks the students, so the 100th percentile (or 99.99th : this always confused me) will be the top student, even if they only got 50% of the answers right.


If you are wondering why I wish to know this. My exam has multiple sections and we get a percentile grading for each section. The total score is then the average of each section. So depending on the answer to my question, it will determine if I study evenly for all the sections of the exam (and aim for say 55th, 55th, 55th), or skew my studying in favour of my strongest subjects to push that mark "quickly" (and aim for say 45th, 45th, 80th) up the percentile curve.

$\endgroup$
  • $\begingroup$ Could someone also please let me know if this would have been a better question for the mathematics stack exchange? I have found the normal distribution tag on both that and this site. $\endgroup$ – K-Feldspar Sep 9 '16 at 9:22
  • 1
    $\begingroup$ For intuition as to the shape of the percentile curve, you could model whether you got a question right as an independent Bernoulli trial. If the probability of answering each question correct were the same, someone's overall score would follow the Binomial distribution (which can be approximated by the normal distribution). $\endgroup$ – Matthew Gunn Sep 9 '16 at 9:31
  • $\begingroup$ Thank you for the response. I read through those links but the answer still is not obvious to me (I couldn't figure out how to do the Bernoulli Trial). However, I did find this 7sage.com/lsat-score-percentile-conversion which shows raw marks vs percentile for another exam. From this it looks as if the opposite to what I thought was true. It shows that when moving from 90 to 100 correct question you go up 2% (as in 2 percentiles) but when moving from 60 to 70 correct questions you go up 22% (as in, 22 percentiles). $\endgroup$ – K-Feldspar Sep 9 '16 at 9:42
  • $\begingroup$ Based on this I would conclude it is INCREASINGLY difficult to move up the percentiles and would require increasingly more correct answers to move up percentiles as you move away from the average (but this seems contradictory to the 2nd graph I posted if I have interpreted it correctly). $\endgroup$ – K-Feldspar Sep 9 '16 at 9:43
  • 1
    $\begingroup$ I deleted my last comment because I was being a complete bonehead (it was totally wrong). If you look at the percentile curve, the slope is highest in the middle. Going from 37 to 39 right or from 73 to 75 right has less of an effect on your percentile than going from 57 to 59 right. Getting an extra question right has the greatest value on sections at which you are average. $\endgroup$ – Matthew Gunn Sep 9 '16 at 10:01
4
$\begingroup$

Some understanding of the why can be gleaned from a simple but realistic model.

The curve shown in the question is consistent with a 46-question test in which each question contributes $100/46 \approx 2$ to the total score when answered correctly and otherwise contributes nothing. It is "consistent" in the sense that the distribution of scores is extremely close to what would obtain if each student were to be guessing each question independently, with a $54.5\%$ chance of being correct and $100-54.5 = 45.5\%$ chance of being incorrect.

Consider some circumstances near the end of the administration of the test. You have answered all questions; you do not know your score; but you are contemplating changing some answers.

  1. Suppose your score (unbeknownst to you) is at the middle, equal to $54.5$. This corresponds to a raw score of $54.5\% \times 46 = 25$, indicating you got $25$ questions right and $46-25=21$ wrong. If you were to pick a question randomly and change it, there would be a $25/46 = 54.5\%$ chance it is correct--and you would turn your answer into a wrong one--and only a $45.5\%$ chance it is incorrect and you would turn it into a correct one. Therefore it's a little bit harder to increase your score than to decrease it.

  2. Suppose your score actually is high, equal to $65$: that is, $30$ correct and $16$ incorrect answers. Now your chance of alighting randomly on one of the incorrect questions and changing it--thereby improving your score--is only about $1/3$. It is twice as hard to increase this high score than to decrease it.

  3. Conversely, using a similar analysis, it is easier to improve a low score by randomly changing one of the answers.

More generally--and you might find this to be a more appealing model than one that seems based on luck alone--consider any test in which your score is expected to be $100p\%$ of the total based on your underlying knowledge. To improve your expected test score from $100p$ to $100(p+x)\%$ -- that is, an increase of $100x$ points -- you would have to retain your performance on the $100p\%$ of the answers you got right while learning enough to add $100x$ points out of the $100(1-p)$ points lost on the wrong answers. This relative improvement in your knowledge can be expressed in two ways:

  1. You reduced the proportion $1-p$ of wrong answers to $1-p-x$, a change of $-x/(1-p)$; and

  2. You increased the proportion $p$ of right answers to $p+x$, a change of $+x/p$.

The ratio of these (up to sign), namely

$$\frac{xp}{x(1-p)} = \frac{p}{1-p}$$

is the odds of $p$. In a balanced way--by accounting for the need both to get fewer wrong answers and more right answers--it measures how difficult it is to make a small increase of $100x$ starting with a score of $100p$. As $100p$ grows towards $100$ points, the dwindling size of the denominator $1-p$ shows how it gets progressively much more difficult to improve an already high score. Roughly, increases from $90\%$ to $95\%$ to $97\%$ are equally difficult. (These are odds of approximately $9$, $19$, and $32$, respectively.)

Note, too, that it's far more likely for your score to drop due to small errors on questions than to rise when your score is above 50%, with the reverse being the case for lower scores: guessing and random mistakes benefit the poor student and hurt the good student.

As far as a study strategy goes, this analysis suggests you get the most benefit from studying the sections you are weakest at--assuming that each unit of study effort results in the same relative increase in performance in each section.

$\endgroup$
  • $\begingroup$ Thanks @whuber, that is very interesting. May I asked why you chose 100/46≈2 instead of 100/50=2, i.e. a 46 question test instead of 50? What is the significance of 46? $\endgroup$ – K-Feldspar Nov 24 '16 at 5:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.