Suppose we are cross-validating parameters of a Gaussian (radial) SVM on $k$ training observations. The parameters are the cost parameter $C$, and the deviation parameter $\gamma$.
Then, $4k$ more training observations arrive. If we train the SVM on the whole training set with the previously validated values $C,\gamma$ this will not be a good practice for obvious reasons. However, if we cross validate $C,\gamma$ on the training set on $5k$ observations, this will be too expensive.
Is there any way in between these two extreme practices, e.g., using directly parameters that are cross validated on the first $k$ observations versus cross-validating the whole $5k$ observations training set?
I would say, since we now have a lot more observations than before, we can instead assume the new $4k$ observations are the validation set. However, I am looking for an answer which is directly from the SVM practice. Maybe there is a clever way.