Indicator Variables for Longitudinal Data

Question

Suppose I have the following question:

• A school has yearly (historical) data on the height and weight of students - each year, students take a fitness test in which they either pass or fail ("score"). Some students have been at the school longer than other students and therefore have more data (randomly simulated data below):

    id   height   weight score
   1 149.9729 76.91511     0
   1 146.3454 84.32803     1
   1 142.6036 73.85985     0
   1 151.2100 80.61091     0
   1 153.9683 82.87161     0
   1 141.9738 75.41321     1
   2 146.1148 68.28539     1
   2 147.5398 76.08225     0
  

• The school is interested in building a supervised classification model that can attempt to predict if an individual student is expected to pass or fail this test based on this data (i.e. based on patterns on the individual student's data and the population of students)

I had the following question: If an indicator variable is added that look at each individual students running average over all years (i.e. percentage of time they passed the test until that time ) and a statistical model such as Random Forest is then fitted to this data - is this a statistically "valid" approach?

   id   height   weight score   average
    1 149.9729 76.91511     0 0.0000000
    1 146.3454 84.32803     1 0.5000000
    1 142.6036 73.85985     0 0.3333333
    1 151.2100 80.61091     0 0.2500000
    1 153.9683 82.87161     0 0.2000000
    1 141.9738 75.41321     1 0.3333333
    2 146.1148 68.28539     1 1.0000000
    2 147.5398 76.08225     0 0.5000000

library(randomForest)
rf <- randomForest(score~., data=data)
pred = predict(rf, newdata = test_data)

One possible concern I thought of is how "valid" this approach would be for students who have only been at this school for a single year (i.e. they only have their height and weight measurements, but no "average" and "score" information) - in this case, I considered setting the first "average" as 0.5 to represent a "neutral probability", and this average would be updated as these students take the test in subsequent years.

Can someone please comment on the statistical "validity" of this approach?

Thanks!

Answer 1

IIUC, you want to add to the features (id, height, weight) as an additional feature the average of the student's performances to better predict the student's next score.

The problem is that, in the training data, the average already contains, in a noisy way, the response, and even future scores. I.e. you use information that is not available when you apply the model. E.g. for students that have taken the test exactly once, the average would contain precisely the score. That might lead random forest to put more "weight" on the "average" feature than it deserves.

You could remedy this by creating training data that contains the average of only the previous scores of this student.

For new students, I suggest taking as imputation, instead of 0.5, the average score of all students with comparative height and weight.

---

Another thought is the following: The average does not contain any trend. E.g. the score from last year is probably more important than that from five years ago. Let's say the maximum number of years at the school is six. Then I would suggest adding the previous five scores as additional data, such that e.g. all the most recent scores are in the same column, as are all the scores from two years ago, and so on. To handle the scores that are not available, because the student was not yet present, you could add additional five binary variables indicating "presence". I trust that random forest will figure out that the value in "score" does not contain information if the belonging field "presence" for that year is zero. This way, you add a small binary time series to each row, which contains considerably more information than just an average.

Also, you don't have the trend in the height-weight ratio. I.e., if you consider this relevant, you could do the same with the two features "height" and "weight" as with the "score", except that those time series would also contain the values for the present.