I have a supervised classifier model (regularized discriminant) which predicts the probability of an event occurring within two years.

This model was developed using sensor data measured from a baseline assessment and followed up for the occurrence of an event two years later (outcome is a binary variable, 'Y' or 'N').

I wish to validate this method using a separate, independent test set. This independent test set contains the same baseline sensor data (measurements) but follow-up was recorded after one year (rather than two).

My question has two parts:

  1. How should I excpect this shorter follow-up period to affect the models performance? (i.e. should I expect lower sensitivity for prediction of events, as events may not have happened yet within follow-up period)?
  2. Is there a way to correct for this shorter outcome period (e.g. treat all outcome data as having equal hazard of 0.5 probability and therefore randomly label half of data points labelled 'Y' as 'N')?

If you assume your events occurs in a poisson process, then the waiting time until an event is exponentially distributed with density function $$ f_Y(y) = \lambda e^{-\lambda y} \qquad \text{for}~~ y>0 $$ ($\lambda>0$, the expectation of $Y$ is $1/\lambda$). Then the probability of an event in no more time than $a$ years is (a simple integral) $P(Y \le a)=1-e^{-\lambda a}$. Calculate this with $a=1$ and then with $a=2$ and you get your correction factor.

You must of course think about IF this is a defensible assumption!

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.