The following is not restricted to NB + LogRes
Overfitting = Loss of generalization.
When you train a model on dataset you generally assume that the data you use for training has a similar structure than the data the model is applied to later (the general assumption of predicting the future from the past). So if you remove parts of the data (namely the misclassified instances) and train a model on this reduced dataset, you effectively change the structure of the data in comparison to the test dataset (and hence violate this assumption). In this case the following can happen (when testing this model on an unreduced test-dataset):
In the best case nothing happens, e.g. of the following reasons:
- The missclassified instances represented only a tiny subspace of the dataspace (corresponds to a high accuracy achieved by the first model)
- The model classifies one part of the dataspace better and another one worse so that they even out.
In the worst case the quality decreases rapidly, because of overfitting / loss of generalization power. The model focuses too hard on the part of the dataspace of in first step correctly classified instances and hence is not able anymore to make even an approximate statement for the rest of the dataspace.
I think what you are actually looking for is called Boosting, where one restricts the dataspace to the missclassified instances (i.e. doing the opposite of your strategy) to refine the model. The procedure tries to avoid overfitting by combining the different (subspace-)models afterwards, but nevertheless it is still an issue.
Here is an plain-text explanation of boosting with a illustrative graphic you might find helpful.