As I understand, most statistical and machine learning models are like "paper airplanes" in some sense: you can spend as much time as you want creating the model, but once you "launch" the model - you can not make any improvements, repairs or corrections to the model until it "lands". For instance, if you train a Random Forest or XGBOOST model, after being deployed these models will fundamentally remain unchanged until you re-train them.

The same way, are there any statistical or machine learning models that have the ability to ACTIVELY "self-improve", "self-repair" or "self-correct" once they are actually in the "air"?

  • The first thing that comes to mind are Stochastic Time Series Models. For instance, models such as ARIMA and Kalman Filters have the ability to change their predictions on current observations based on previous observations.

  • I am also thinking about Gaussian Process models (e.g. Gaussian Process Regression) in which during the "training period", each new observation has the ability to influence the Gaussian Process via the "Posterior Update" - but this requires the observation to have a "response label" associated with it. If I have understood this correctly, this means that when the Gaussian Process model is deployed, since new observations by definition will not have a corresponding "response label" - the "Posterior Update" can not take place and the Gaussian Process model will remain identical to what was created during the "training period".

  • I have heard that Reinforcement Learning models have the ability ACTIVELY "self-improve", "self-repair" or "self-correct" once they have been deployed - but I have only heard about this anecdotally and can not confirm if this is the case or not.

My Question: Can someone please tell me if there exist any statistical or machine learning models that have the ability to ACTIVELY "self-improve", "self-repair" or "self-correct" once they have been deployed?


1 Answer 1


This is a question of the data fed to the model when it is deployed. For example, in Netflix's recommendation system, a model suggests you the movies you are likely to like. Say you watch a suggested movie and then you rate it, as you see the model can keep training itself, to minimise the error between your rating and its recommendation score of this movie. In other circumstances , specifically in the supervised learning context, you don't have such feedback (user ratings), so the model can't be improved.

In the unsupervised learning scheme, I think the model can make use of its input and improve itself, whether when it is trained or when it is deployed.


Not the answer you're looking for? Browse other questions tagged or ask your own question.