i am training a SVM (RBF kernel) with a dataset of ~1500 samples (balanced) using fminsearch on the CV error for parameter optimization (C and s).
After i found the "best" parameters (local optima possible) i am retraining my model on the whole dataset to derive a "final" model.
Is this a wise thing to do? Would the final model be proned to overfitting?
I experience worse performance on unseen data which might be OK as the CV during my optimization approach produces a somewhat optimistic estimate on the test error.
I think this adresses a pretty general problem but i could not find proper reasoning yet... Would it be a reasonable alternative to use just one of the models from the best performing crossvalidation?
I assumed that once the parameters are fixed the model will not suffer from overfitting no matter how many more samples i use for training?