This is sometimes called "concept drift:" when you're developing the model and deploying it, the scores do a nice job of matching the true values, but over time, the scores and true values start to diverge.
Concept drift arises in lots of contexts that have a time component that is not explicitly modeled. For example, new malware will be invented to evade detection, so a malware classifier using machine learning will need to be refreshed. It would be nice to have a model that automatically adapts itself, but solving that problem is a much harder task than just building a new model on some schedule.
I'm not sure that you can learn anything more specific from the observation that you're experiencing concept drift.