I have a classification problem where neural networks appear to be a promising choice (I don't get into details, because my question is about the general approach)
When using "classical" machine learning algorithms, a common solution for multi class classification is to train a classifier for each class (one vs. all) - this has the benefit of easily adding and removing classes.
Theoretically, this can also be done with neural networks, but then we loose the benefit of "reusing" the existing neural network that was trained for the other classes
are there any "best practices" for this type of problem?
My basic ideas are (they are basic ideas - not "production ready"):
- use one vs all training, and pick an existing network as template
- use a network with one output neuron for each class, and adding an output for each new class (which requires changing the network structure)
- use an auto encoder that trains on all data, and build a "classic" classifier based on the output of the auto encoder hidden layer