I have some data that represents students exam scores, this data is used to predict what rank the student will be in 'todays' exam. Some of the data I use in order to predict this is the previous exam scores that these students achieved. Unfortunately the previous exam scores are not always from the same subject as todays exam. For instance todays test may be calculus and yesterdays (or the last one taken) may have been Latin.

If I can describe my previous standardisation methodology to help add some context. First thing to note is that I am not currently trying to work out what test score a given pupil should achieve for this exam they are taking, I am trying to calculate where they should rank against their peers within that exam class.

  • My original method: I calculated what the average score in a calculus exam was by a student that achieved a 98 in the Latin exam (which he took as his last exam) via a simple non linear regression. I do this for all the students sitting a particular exam, calculating what I think their previous exam score would be worth if it was in todays subject. I even do this for same subject scores, the reason being when a student scores a ridiculously high score one exam, in reality on average they achieve slightly lower in the next exam (according to my regression.)
  • What I then do is find the difference between the largest last time exam score for a class and the last time exam score of every pupil taking todays exam. e.g. The highest scoring pupil will have a value of 0 and those who achieved worse will have a positive number reflecting how far away they were from the person who achieved the best score, lets call this value deltaE for reference purposes.
  • Then I calculating subsequent percentage exam class win of different deltaE values, for example those with a deltaE of 0 are found to have a 18% chance of ranking highest in their next exam.
  • I then use this data in an attempt to predict where a student will rank using a random forest classifier, including all three parameters 18,100,0.18 representing amount of times a deltaE of 0 rank top in their next exam, from a total number of deltaE-0 students and finally the 18/100 represented as a decimal

This method of standardising the different subjects into an expected subject relevant score to me seems messy and more than likely the long way of doing things. I am wondering if it is possible for me to use the random forests functionality to remove the need for this standardisation? For instance could I use a numerical representation of each subject and include this in my data? For instance Latin could be 18,100,0.18,2 however if the subject was calculus it would be 18,100,0.18,1 or something to this effect.

All comments and suggestions welcome, someone with experience of random forests would be ideal to comment on this, but if your experience is in other modelling programming backgrounds feel free to put your thoughts across. Thanks in advance

up vote 1 down vote accepted

You can add categorical variable which will represent the subject, RF should be able to "figure out" what is the correlation of score between the particular subjects is. Instead of just one variable you can have multiple variables, one for each subject and assign 0 or 1 to each. Yet another option to prevent too many variables to be created - split subjects by groups (technical, humanitarian, languages, etc...), again, creating a 0/1 binary variable for each.

The problem with only one categorical variable is it could be confusing for RF because splits do a comparison. What would subject < 4 mean? It doesn't make much sense... It still could work, but I believe binary variables should work better (almost?) always.

All in all, which of these ways works better in your case can only be tested on practice, given your constraints on how quickly you need to train and what precision do you need.

Your Answer

 
discard

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.