Let's assume the following: I successfully trained a neural network on a classification task, it performs well, also on unseen data.
Now my idea is: If the neural network obtains new, unseen data and classifies it, can I add this data to the training set? Via increasing my training set, I want to improve the neural network's performance, as it will always learn new examples, and not stay the way I trained it.
The main problem that I see with this approach is the risk of misclassified samples. They will be rare, but they might occur. Will a few misclassified samples in the training data badly harm my network?
I know, this question is pretty case-dependent, and I don't expect a finite answer. But I didn't find anything to it online, so I'm glad about every hint, if that idea might work out or not.