How to build a model (or a set of models) when number of independent variables (features) are changing over time? Suppose we are building an "early warning system", that will output a risk score every month over 1 year, say Jan. to Dec..
The tricky part is that number of available variables/columns will be different (increase over time). For example, at beginning (Jan.), we only have $5$ features/columns for each instance, every month we will have more features. So, by December, we will have say hundreds of features. 
What is a good practice of dealing with such data? Should we build $12$ different models that only work for specific month? Building $12$ models and switching among them would be a messy implementation. Is there any better way of doing this?

Here is an example of data I was talking about:


*

*Month 1: Student age, gender, income, ...

*Month 2: all month 1 features/columns + Student math unit 1 test score, science unit 1 test score, ...

*Month 3: all month 2 features/columns + Math GRE score, Writing GRE score ...

*Month 4: all month 3 features/columns + Survey response data ...
The idea is that different types of the data is only available at certain time of the year. In above example, student can only take unit test 1 at Feb. and GRE at March, and give survey response at Apr. etc.
 A: I think a cross between a neural network and a vanilla RNN would work for this. 

predict1                                  predict2 
     |                                                   |
[Module1] ---h1 ---> [Module2] --->h2--->....
    |                                        |
    |                                                          |
   x1                                                     x2

So each xi is relative to each month, meaning that if you add features, xi, will grow in size. Each module's $m$ weights are relative to the set of features available in that month $m$, and the memory input of the last module. So unlike an RNN, the modules change with each extra month of data.
Note that you can grow the size of your memory hi, to accomidate for larger xi. To train the model, for each data point you would first identify how many months $m$ of data you have, and then apply the modules $m$ times, to get a sequence of $m$ predictions, which you can then evaluate using whatever error metric you want.
A: As it has been already noted building $P$ distinct models for each of the $P$ periods is probably an overkill both in terms of training as well as  deployment. I think there two major points that must be considered:


*

*treating the initial question as producing a probability for a student to "churn" (to use a standard marketing term) during her life-cycle.

*treating the "new features" as additional information to existing features rather than stand-alone new pieces of information.


These points combined culminate to the reformulation of the original problem as one where the number of features does not change over time; we have a fixed number of features that potentially evolve across time.
I briefly elaborate each point separately: 
Point 1. To start with, the fact we have some predefined periods is somewhat immaterial. In every time-point from enrolment to graduation a student has a "probability to churn". We should treat the period as one more feature. It is extremely unlikely that the baseline probability to drop-out is constant across periods, as such the "academic age" is an important feature. (Probably using absolute days is even better) Just with that we move the goal-posts from $P$ models to $1$. If our classifier can handle missing values we are sorted. On to the next step!
Point 2. We must consider what the features at hand show. You write: "Student math unit 1 test score (...), Math GRE score, (...)"; what I read is: "Maths aptitude". We can use the standard metrics for short-time series prediction; last value, median, linear trend, etc. we can dress it up as we like, but bottom line is we measure Maths aptitude. We have multiple aptitude indicators for subject $X$ that inform our general $X$-specific features as time evolves. The same extend to survey-response data. Students always have positive or negative attitudes towards some aspect of their courses and these evolve over time.
Point 2+1. (Some extra sub-points deriving from above) The missingness mechanism is probably very important (eg. students who miss exams are almost certainly more probably to drop-out than the ones who never missed an exam and this is probably an additive effect); this will be a pain to include in $P$ different models but extends naturally from points 1 & 2. Similarly, "market conditions" and seasonality matter (eg. students who got A's when everyone else got A's or B's are probably not as strong as the one's that got B's when most the class scrapped C's and D's).
In general I have the slight impression you jumped from EDA straight to learning and did not labour enough the feature engineering part of the question.
