I have a dataset of performance of students in exams which looks like:
Class_ID Class_size Student_Number IQ Hours Score
1 3 3 101 10 98
1 3 4 99 19 80
1 3 6 130 3 95
2 5 4 93 5 50
2 5 5 103 9 88
2 5 8 112 12 99
2 5 1 200 10 100
2 5 2 90 19 78
3 2 5 100 12 84
3 2 7 102 13 88
and I would like to build a machine learning model trying to predict who is going to be top of the class (i.e. highest Score
) for any given Class_ID
using IQ
and Hours
(number of hours studied) as features.
In other words, the input is IQ
and Hours
for every student (say 1 to n) in the class and the output is the probability vector (p_1, ..., p_n) where each p_i is the probability that student i have the highest score in the class.
Here is what I have tried:
- Since this is a ranking problem, a natural class of learning models is to use the
XGBRanker
inXGBoost
orLGBMRanker
inlightgbm
. Unfortunately, the output is a list ofrelevance score
as opposed to a list of probabilities which have no natural interpretation in terms of probability.
One way to get around it is to apply softmax on the relevance score in xgboost, but there is no direct meaningful probabilistic interpretation like in energy based models such as RBM's energy function. In fact, I have tried to do it and the probabilities become very extreme (most probability mass concentrates on one student for every class, giving a poor test result with high variance)
- Another class of learning models is the classification model like logistic regression/ decision trees. However, the trouble I have is that the number of students in each class is different, hence to train such a model we will have to first "flatten" the feature matrix:
Class_ID Class_size IQ_1 IQ_2 IQ_3 IQ_4 IQ_5 Hours_1 Hours_2 Hours_3 Hours_4 Hours_5 Score_1 Score_2 Score_3 Score_4 Score_5
1 1 101 99 130 NaN NaN 10 19 3 NaN NaN 98 80 95 NaN NaN
2 5 93 103 112 200 90 5 9 12 10 19 50 88 99 100 78
3 2 100 102 NaN NaN NaN 12 13 NaN NaN NaN 84 88 NaN NaN NaN
such that that 1 row represents 1 training example. But then the feature matrix becomes very sparse (as the number of students in different classes can be very different)
Hence logistic model/ vanilla feedforward neural net/ tree based model doesn't seem to be natural to these kind of group data either.
So my question is, Is there any natural class of machine learning models to deal with these "group data" like the dataset I have above?
Also, in this problem, I only care about the person who ends up the top of the class (or maybe the top 3) so the ranking isn't all that important (for example knowing student 4 ranks 11th and student 8 ranks 12th does not matter all that much)
Thank you so much in advance.