How to measure student skill level based on a set of questions answered at different points in time and of different difficulties levels? I am working on an e-learning system with a friend for our final year (Computer Science) project which is part of the under-graduate programs mandatory 'courses'.
I have a question about making inferences and therefore gauging the skill level of the 'subject (student) at hand'
We have gathered the following data thus far for every attempt made at answering a question:


*

*Unique ID of the question

*Attempt number

*Outcome of the attempt (Correct/Incorrect)

*Difficulty of the question (assigned a nominal score between 1-5)

*Date and Time at which the attempt was made


How can I use the entire data set regardless of what point in time the answers were given and judge the skill level of the student from that particular model?
I know I could just look at the last X number of attempts for a section of a particular difficulty level and pick up a trend from that, but is there a better way to do this?
 A: The easiest way to determine skill level would be to simply add up the number of questions correct for each student on the very first attempt. Unless you have a clear reason why attempt count is important, it is vague whether number of attempts would represent skill level. Plus, item difficulty is taken into account even if you disregard it; students who get the difficult questions right will simply get more points. This is the way most tests are assessed. 
You should try calculating the reliability too. look up kuder-richardson 20. The idea is that students who get high overall scores should be the ones that are more likely to answer the difficult questions (ones that not many get correct) correct, and vice versa. If the reliability is low, it means your test was poor at discriminating students who have good/bad knowledge, which is the whole point of the test. 
A: There are quite a number of elaborated test theories for estimating person paramters based on test data and soemtimes simultaneouls estiamting item difficulties. I wouild look at Item Response Theory and Rasch Scaling procedures. They might not cover your particular concern about time points, though. I would argue that no statistical model will be able to capture how much learnign is going on between time points, unless you build a psychological model of learning trajectories that is precise enough to make valid projections. If you assume hetereogeneity in learnign progress, you might not have enough data to assess learning rate and skill level simulatenously while still controlling for error rates....
But I would first commit to a test theory, and then define the measurement problem and repeated-testing and learning problem. 
A: I would like to add to @jank's answer. I also think that item response theory is the way to go. If you haven't heard of it, yet, there might be a bit of a learning curve. But if you are really interested in test theory, you should definitely have a look at it, as it entails a lot of the advances that have been made in this area. A good and detailed book, that is freely available is that by Reckase (2009). 
The article about the eRm-package for R also has a nice introduction. What's more important, there is a section about using IRT for longitudinal data, which is of relevance to you.
References:

Reckase, M. (2009). Multidimensional item response theory. Springer.
Mair, P., & Hatzinger, R. (2007). Extended Rasch modeling: The eRm package for the application of IRT models in R. Journal of Statistical Software.

