Training an SVM and performing cross validation I am training an SVM and I have 40k Negative Samples and 17k Positive samples. What I did is that I have divided my samples into training and testing subsets. In order to train the SVM I have used some of the training (not all) and I was randomly picking samples and apply the SVM into all testing data. I have found that if I use a specific amount of training data (not all) I get a very good performance over the testing data (0.99 sensitivity and 0.99 specificity).  
How do I use k fold cross-validation now? I am confused. As I understood it, in k fold all the available data are used and they are divided in 5 subsets etc. Which will be the final SVM that I will use in 'real time'? The one of that I have found with my own good results? 
I am using MATLAB (svmtrain, svmclassify, classperf).  
 A: I do not think that is the standard order of operations in cross-validation and you may have overfitting on the test set. Briefly, I believe that the standard practice is to:


*

*Separate the testing set and put it to one side.  Treat that testing data as the equivalent of an as-yet-unknown future which you will test your final model against.

*Partition your training set into the $k$-folds, so if $k=5$ then split the training set into five parts.  

*Train your model (in effect decide hyperparameters, perhaps in your SVM the soft constraint parameter $C$ and the kernel parameter $\gamma$) by looking at how it performs with different hyperparameters when you run it on the combination four parts and validate against the fifth part - each run doing this five times,  against each of the folds. Choose the hyperparameters which give the optimal results in this cross-validation.

*Run your model, using the chosen hyperparmeters, on the whole training set.  This is your final model.

*Test (once only) your final model on the test set to see how accurate/sensitive/specific it is on what is designed to represent out-of-sample data.

