I'm working on an adaptive learning platform with some friends for a project that helps students to prepare their university admission test.
We provide different kind of practice sessions and we have some millions of answers given. Each question has 5 possible choices and we gathered the following data for each attempt made at answering a question:
- Question ID
- Answer ID
- Outcome of the attempt (correct / incorrect / skipped)
- Time taken to answer
My question is: how can I use the entire data set to rate the difficulty of a question?
My idea was to normalise each data removing answers given in too less or too much time and judge the difficulty using a float number between 0 (very easy) and 1 (very difficult). But how?