I have a supervised classifier model (regularized discriminant) which predicts the probability of an event occurring within two years.
This model was developed using sensor data measured from a baseline assessment and followed up for the occurrence of an event two years later (outcome is a binary variable, 'Y' or 'N').
I wish to validate this method using a separate, independent test set. This independent test set contains the same baseline sensor data (measurements) but follow-up was recorded after one year (rather than two).
My question has two parts:
- How should I excpect this shorter follow-up period to affect the models performance? (i.e. should I expect lower sensitivity for prediction of events, as events may not have happened yet within follow-up period)?
- Is there a way to correct for this shorter outcome period (e.g. treat all outcome data as having equal hazard of 0.5 probability and therefore randomly label half of data points labelled 'Y' as 'N')?