What I mean is the following: Instead of processing all the training data at once and calculating a model or hypothesis, we process one data point at a time and update the model directly afterwards.
I have seen the terms "on-line (or online) learning" and "incremental learning" for this. Is there a subtle difference? Is one term used more frequently? Or does it depend on the research community?
The Bishop book ("Pattern Recognition and Machine Learning") uses the terms on-line learning and sequential learning as synonyms but does not mention incremental learning. Are they really synonyms or is there also a small difference?
Update: I have received interesting answers for the same question here at datascience Stackexchange.