1
$\begingroup$

I have a dataset of performance of students in exams which looks like:

Class_ID   Class_size   Student_Number   IQ   Hours   Score
1          3            3                101  10      98
1          3            4                99   19      80
1          3            6                130  3       95
2          5            4                93   5       50
2          5            5                103  9       88
2          5            8                112  12      99
2          5            1                200  10      100 
2          5            2                90   19      78
3          2            5                100  12      84
3          2            7                102  13      88

and I would like to build a machine learning model trying to predict who is going to be top of the class (i.e. highest Score) for any given Class_ID using IQ and Hours (number of hours studied) as features.

In other words, the input is IQ and Hours for every student (say 1 to n) in the class and the output is the probability vector (p_1, ..., p_n) where each p_i is the probability that student i have the highest score in the class.

Here is what I have tried:

  1. Since this is a ranking problem, a natural class of learning models is to use the XGBRanker in XGBoost or LGBMRanker in lightgbm. Unfortunately, the output is a list of relevance score as opposed to a list of probabilities which have no natural interpretation in terms of probability.

One way to get around it is to apply softmax on the relevance score in xgboost, but there is no direct meaningful probabilistic interpretation like in energy based models such as RBM's energy function. In fact, I have tried to do it and the probabilities become very extreme (most probability mass concentrates on one student for every class, giving a poor test result with high variance)

  1. Another class of learning models is the classification model like logistic regression/ decision trees. However, the trouble I have is that the number of students in each class is different, hence to train such a model we will have to first "flatten" the feature matrix:
Class_ID Class_size IQ_1 IQ_2 IQ_3 IQ_4 IQ_5 Hours_1 Hours_2 Hours_3 Hours_4 Hours_5 Score_1 Score_2 Score_3 Score_4 Score_5
1        1          101  99   130  NaN  NaN  10      19      3       NaN     NaN     98      80      95      NaN     NaN
2        5          93   103  112  200  90   5       9       12      10      19      50      88      99      100     78
3        2          100  102  NaN  NaN  NaN  12      13      NaN     NaN     NaN     84      88      NaN     NaN     NaN

such that that 1 row represents 1 training example. But then the feature matrix becomes very sparse (as the number of students in different classes can be very different)

Hence logistic model/ vanilla feedforward neural net/ tree based model doesn't seem to be natural to these kind of group data either.

So my question is, Is there any natural class of machine learning models to deal with these "group data" like the dataset I have above?

Also, in this problem, I only care about the person who ends up the top of the class (or maybe the top 3) so the ranking isn't all that important (for example knowing student 4 ranks 11th and student 8 ranks 12th does not matter all that much)

Thank you so much in advance.

$\endgroup$
2
  • 1
    $\begingroup$ Why not just predict the final score and do the ranking afterwards yourself? $\endgroup$ Commented 16 hours ago
  • $\begingroup$ @user2974951 because it suffers from a similar problem with relevance score: There is no natural way to convert the predicted score into probabilities, and what I want is the probability distribution of a student getting the highest score in any given class. $\endgroup$
    – Ishigami
    Commented 16 hours ago

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.