I am using SVM classifier to classify data, My dataset consist of about 1 milion samples,
Currently im in the stage of tunning the machine , Try to find the best parameters including a suitable kernel (and kernel parameters), also the regularization parameter (C) and tolerance (epsilon).
My corrent approach is using a blackbox global optimization algorithm to find the best parameter set, i use k-fold cross validation as a minimazation function, The optimization algorithms i have in stock: Cma-ES, Simplex, HillClimbing, Down hill, GA and Simulated Annealing.
The problem is that the crossvalidation is a very slow process which makes all of the algorithms run for hours or even days.. and from what i know crossvalidation is the only option i have..
I want to know if it's possible to run the optimization on only a small part of the dataset to improve the runtime? , or are the parameters i will find will not work well on the full, larger dataset.
Also are the kernel parameters have any correlation with the C parameter? is it possible to firstly tune the C parameter and only after one was found continue and optimize the kernel parameters?
I know hyperparameter tuning is a very common issue so how is that im feeling there is no "clean" solution for this problem.. It must be a way that makes it possible for large datasets,
Ill appreciate any kind of help and advices