# After training a classifier on N classes, how do you add an N+1th class?

Say we have data for N classes and train a classifier. Then we have a new set of data for a N+1th class. How do I train a classifier that now predicts all N+1 classes?

Setting is in object detection, though I don't think it should be different between image classification and object detection. Also, I am finetuning from a pre-trained model (e.g. VGG16 trained on image-net).

Option 1: Compile all the data for the N+1 classes and finetune again from the original VGG16 trained on image-net.

Option 2: Compile all the data for the N+1 classes and finetune from the model trained on N classes.

Option 3: Is there a way to finetune only using the data from the N+1th class? I assume no, since the last layer's weights (right before class prediction) need to be re-initialized.

Options 1 and 2 aren't ideal b/c it isn't scalable as we increase the number of classes.

Another approach is to train autoencoders for your first N classes, and then train another network $f$ which maps trained autoencoder weights to vectors $W_i$, where $W_i$ is the $i$th row in the weight matrix in the FC layer before the softmax (this vector would correspond to the $i$th class being picked).

Now in order to add a new class, you can simply train an autoencoder on just that class, and use $f$ to predict the vector $W_{N+1}$. Now you can concatenate it onto your weight matrix, and perhaps fine-tune a bit.

This doesn't require retraining on all the data (although finetuning may improve performance), and the cost is constant with respect to $N$.

Also see the extremely relevant paper "Learning to Model the Tail"

Option 2.

Model generates (presumably) some kind of Softmax output in the last layer. You will have to train the network to distinguish class N + 1 from all other classes. So to the best of my knowledge, there's no magic to just incrementally train on new class.

I'd treat output of VGG as a unsupervised pre-processing of input. That need not be tuned again for new data classes. In other words, this is the goal of pre-trained models: unsupervised learning usable across applications.