I perform multi-class classification using a Deep Neural Network. I split the data set into training, validation, and test set. To create a strong classificator, I train multiple models using various parameters (e.g. learning rate). Then I compare the results of each trained model on the test set, choosing the model that performed best.
Is this a valid approach? Or should choosing a model based on results on the test set be seen as an optimizing process, which might be biased towards the specific test set and thus not generalize well (overfitting)? Basically, am I violating the independence of the test set with this approach? If so, what would be the correct way to split the data with this approach?