I have a classifer (one/zero labels) that was trained and hypertuned by the book. When the model was ready, I applied it to the production data: real-time and unlabeled.

After a short period (a few days); the real-time data became historic data and labeled. I checked the model real-time data (now, historic data) predictions, and analyzed them using LIME. If I see that a specific feature many times involves in FP or FN predicts and I decide to remove it.

Is it a good strategy or do I create an overfitting situation?

  • $\begingroup$ +1, nice question and good intuition on "an overfitting situation" as I read it you were thinking of catastrophic forgetting/inference. See my answer below for more details. $\endgroup$
    – usεr11852
    Commented Nov 23, 2022 at 3:43


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.