As it has been already noted building $P$ distinct models for each of the $P$ periods is probably an overkill both in terms of training as well as deployment. I think there two major points that must be considered:
- treating the initial question as producing a probability for a student to "churn" (to use a standard marketing term) during her life-cycle.
- treating the "new features" as additional information to existing features rather than stand-alone new pieces of information.
These points combined culminate to the reformulation of the original problem as one where the number of features does not change over time; we have a fixed number of features that potentially evolve across time.
I briefly elaborate each point separately:
Point 1. To start with, the fact we have some predefined periods is somewhat immaterial. In every time-point from enrolment to graduation a student has a "probability to churn". We should treat the period as one more feature. It is extremely unlikely that the baseline probability to drop-out is constant across periods, as such the "academic age" is an important feature. (Probably using absolute days is even better) Just with that we move the goal-posts from $P$ models to $1$. If our classifier can handle missing values we are sorted. On to the next step!
Point 2. We must consider what the features at hand show. You write: "Student math unit 1 test score (...), Math GRE score, (...)"; what I read is: "Maths aptitude". We can use the standard metrics for short-time series prediction; last value, median, linear trend, etc. we can dress it up as we like, but bottom line is we measure Maths aptitude. We have multiple aptitude indicators for subject $X$ that inform our general $X$-specific features as time evolves. The same extend to survey-response data. Students always have positive or negative attitudes towards some aspect of their courses and these evolve over time.
Point 2+1. (Some extra sub-points deriving from above) The missingness mechanism is probably very important (eg. students who miss exams are almost certainly more probably to drop-out than the ones who never missed an exam and this is probably an additive effect); this will be a pain to include in $P$ different models but extends naturally from points 1 & 2. Similarly, "market conditions" and seasonality matter (eg. students who got A's when everyone else got A's or B's are probably not as strong as the one's that got B's when most the class scrapped C's and D's).
In general I have the slight impression you jumped from EDA straight to learning and did not labour enough the feature engineering part of the question.