1
$\begingroup$

Definition

An exam is composed of x questions that are taken in a pool of questions (so a question can be in several exams).

Each question gives an amount of points and you can success (gain all the points) or fail (0 points) the question.

I want to compare a student score on a specific exam to the whole community based on question success rate (the whole community hasn't passed this specific exam but can have passed some questions on another exams).

Example

Exam A:

  • Question 1 (1pt) : success rate 75% of students who have passed this question (no matter the examen) have succeed to this question
  • Q2 (10pts): 25% success rate
  • Q3 (5pts): 50% success rate
  • ...

The student passes the exam A and has the following result:

  • Q1: success (gain 1pt)
  • Q2: fail (0pts)
  • Q3: success (gain 5pts)
  • ...

Problem

Is there a way (or an approximate one) to say: "this student is better than XX% of the community on this exam" ?

What I've tried

I tried the following approach but I'm not a mathematician and I don't know if it's legal or not:

  • compute a score for the exam by balancing the question point depending on the success rate:
    • Q1: 75% (success rate) of 1pt = 0.75pts
    • Q2: 25% of 10pts = 2.5pts
    • Q3: 50% of 5pts = 2.5pts
    • ...
    • = (0.75 + 2.5 + 2.5) / (1 + 10 + 5) = 36%
  • compute the score of the student (by cumulating the points of good answers): (1 + 5) / (1 + 5 + 10) = 37,5%
  • try to find the student score on a distribution which is centered on the exam score (36%) and which height is calculated using the standard deviation of the question success rates of the exam
$\endgroup$
  • $\begingroup$ "try to find the student score on a distribution which is centered on the exam score (36%) and which height is calculated using the standard deviation of the question success rates of the exam" This approach is often used in rating formulas, but I consider it a bad practice when quantiles can be explicitly calculated. $\endgroup$ – Viktor Dec 9 '17 at 8:58
  • 1
    $\begingroup$ "75% (success rate) of 1pt = 0.75pts" This approach is unclear. The greater the success rate the easier the question is. It is natural to give more points to more difficult questions. $\endgroup$ – Viktor Dec 9 '17 at 9:03
1
$\begingroup$

If we want to distinguish students who solve hard problems, the score of a student may be $\sum_j \big( s_j f(q_j)- (1-s_j)(1-f(q_j))$, where $q_j$ is the ratio of correct answers to the $j$-th question, $s_j=1$ if the student has solved the $j$-th question and $s_j=0$ otherwise, $f$ is some decreasing function such that $f(q)\in [0,1]$.

For example if a student is given questions Q1,Q2,Q3 with 40%,10%,50% of correct answers and he solved only Q1,Q2 then his score will be $f(0.4)+f(0.1)-(1-f(0.5))$.

For instance, one may put $f(q)=1-q$ or $f(q)={\rm logistic}(c(0.5-q))$ with some constant $c$.

The sense is that the score must increase significantly when the student solves a hard question and decrease slightly when the student fails to solve a hard question. Symmetrically, the score must decrease significantly when the student fails to solve an easy question and increase slightly when the student does solve an easy question.

$\endgroup$
0
$\begingroup$

The simplest approach may be not to consider the total success rate for the questions. One may calculate the percentage of points gained by each student as $P/T$, where $P$ is the total number of points gained by a student, and $T$ is the total number of points of the questions the student tried to solve. Then, for a given student, we may just calculate the percentage of students with lower $P/T$ rating.

For example, let the questions Q1,Q2,Q3,Q4 be of $10,30,5,15$ points, respectively. Student A is given Q1 and Q2, he solves Q1 and fails to solve Q2, so A's score is $10/(10+30)=0.25$. Student B is given Q2,Q3,Q4, he solves Q2 and fails to solve Q3 and Q4, so B's score is $30/(30+5+15)=0.6$. Thus student B is thought to be better than student A.

$\endgroup$
  • $\begingroup$ thanks for the reply, the problem with this approach is that I would like to compare the students to all the students that have passed any question in the exam (and not necessary all the questions of the exam). So I can't compare score like this because this comparison involve only the students who have passed all the questions of the exam, isn't it? $\endgroup$ – Jerome Cance Dec 11 '17 at 8:46
  • $\begingroup$ @JeromeCance No, this approach is to compare all students. $\endgroup$ – Viktor Dec 11 '17 at 13:39
  • $\begingroup$ @JeromeCance Added an example to illustrate the approach. $\endgroup$ – Viktor Dec 11 '17 at 13:47
  • $\begingroup$ The problem with this approach is that if student A only takes easy questions in an easy exam (let's say 0.7), he will have a great score compared to the student B which takes very hard questions in a very hard exam. If the student B get a 0.5 score, he can be a genius because the exam is really hard but just comparing the score does not reflect that. $\endgroup$ – Jerome Cance Dec 12 '17 at 8:28
  • $\begingroup$ @JeromeCance Have I got right that a hard question may be of 10 pts and an easy question may be of 10 pts as well? $\endgroup$ – Viktor Dec 12 '17 at 9:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.