How to determine if a my Support Vector Regression is Overfitting I have a relatively small sample size and I am using cross validation but I am still concerned that my SVR model is overfitting. What method is available to check this?
What should I be looking at or thinking about when figuring if my model overfits using SVR?
Here is some code I used in R to create my model.
tc          <- tune.control(cross = 10, fix = 8/10)
tuneResult <- tune(svm, Y ~ .,data = allData,
                   ranges = list(epsilon = seq(0,.2,0.1),
                                 cost = 2^(2:9)), tunecontrol = tc)

Call:
best.tune(method = svm, train.x = Y ~ ., data = allData, ranges = list(epsilon = seq(0, 
    0.2, 0.1), cost = 2^(2:9)), tunecontrol = tc)


Parameters:
   SVM-Type:  eps-regression 
 SVM-Kernel:  radial 
       cost:  4 
      gamma:  0.125 
    epsilon:  0 


Number of Support Vectors:  19

 A: Cross-validation provides an unbiased estimate of classifier performance on unseen data. Good cross-validation performance usually suggests that you have not overfit your model. I say "usually" because it is still possible to (incorrectly) use cross-validation performance itself to select or parameterize models, in which case the CV performance would no longer be an unbiased estimate for unseen data.
A: There are few things that you may want to check.


*

*Is training error large?

*Is training error << validation error?


In the first case, if the training error is large, this means that you are perhaps dealing with non-linearities. This might be difficult to tell with small sample size. Use linear kernel and then Gaussian kernel (for example) and see how this error changes.
In the second case, if training error is much smaller than validation error, your model may be overfitting. You may want to tune parameters such as C or \nu (depending which SVM formulation you use).
In resume, try to get low training error first and then try to get validation error as close to it as possible. Once you are satisfied, you want to check the error on a held-out set. The error should be in the same ballpark (usually bit higher). If this is the case, you are done.
It is easy to get the first but harder to get both training and validation errors low. Even harder to get the test error low as well.
