Should one merge training and testing sets when predicting a class of a new sample? I have a binary dataset and I want to build a classifier. I understand that to monitor performance I need to split to training/test set and report accuracy or any other metrics that interest me on the test set. Now, I have a new example for which I do not know the label and I want to predict it using my classifier.
My question is: should I use the classifier that I built using the training set only or merge both training and test set, build a classifier based on all available data and then predict my new example while still reporting the accuracy based only on a the test set?
 A: So in your case, what you refer as the test set is in reality the validation set. And your new example would serve as the test set. 
Normally, you would find the best model which minimize the validation error and use that to predict on the new example. Whether or not you retrain on the whole dataset (train + val) is up to you and can depend on the amount of data that you have and perhaps also the amount of time the training takes. But in general more data is better so I would retrain on train+val with the hyperparameters found minimizing the validation error.
A: You should be using only the model built using the training data set. Unless you realise that your training and test data are very similar and there is going to negligible difference when building a model including both, do not add test set to train for building a new model. In case you have a slightly different model, you would not be able to test how it would fare on an out of sample data (cross validation might be an alternative.
When you go into model maintenance, you could start with newer test and train. By this time, it is expected that you have more data, meaning more samples to build a more robust model.
A: This is a fantastic question. Both Tom and Srikrishna make good points. What it comes down to is choosing hyperparameters. Having selected hyperparamaters using a training and validation set, and assessing accuracy using a testing set, it will not be possible to know if those same parameters will work well if you use all your data for training and validation.
I suggest you do a bit of research to determine if data generated from a sufficiently similar distribution as your training / validation data will perform well.
