The incremental learning system comprises of deep and online machine learning models
As far as I understood, in the online model, the initial model is not available and we train a model to be our initial model using online learning. For example deep learning model with SGD. Besides, they state two types of incremental learning: class incremental learning (CIL) and data incremental learning (DIL).
They also state that
Note that the OL model is constructed using features generated from the LSTM model which can be 64, 128 or 256 in length depending on the number of hidden units present in the LSTM model.
So, online learning (OL) model is a separate model? If yes, I do not understand how SGD works in that way.
I, more or less, understood that the class incremental learning is different from online learning because it has new classes which is not the case in online learning. However, I do not understand the difference between online learning and data incremental learning. Maybe using a different loss (e.g. weighted loss) causes to have different learnings but I am not sure. Can anyone give me more explanation about it in the aspect of online, class incremental and data incremental learning?