5
$\begingroup$

What I mean is the following: Instead of processing all the training data at once and calculating a model or hypothesis, we process one data point at a time and update the model directly afterwards.

I have seen the terms "on-line (or online) learning" and "incremental learning" for this. Is there a subtle difference? Is one term used more frequently? Or does it depend on the research community?

The Bishop book ("Pattern Recognition and Machine Learning") uses the terms on-line learning and sequential learning as synonyms but does not mention incremental learning. Are they really synonyms or is there also a small difference?

Update: I have received interesting answers for the same question here at datascience Stackexchange.

Improve this question
$\endgroup$
0

2 Answers 2

Reset to default
2
$\begingroup$

Regarding the first two, online and incremental learning, some authors underline that

On-line has to discard a sample after learning (no memory) and unlike to incremental learning is not allowed to store it. Source: this paper

Improve this answer
$\endgroup$
1
$\begingroup$

To be honest, I have only rarely seen the term "incremental learning" come up. However, I have seen the "incremental" modifier used more often to describe specific out of memory implementations of various algorithms (e.g. incremental SVM, incremental decision trees) fairly frequently.

It seems that use of "incremental" is used synonymously with "online" or "sequential."

Improve this answer
$\endgroup$
1

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.