0
$\begingroup$

My data has 3168x7 (being targets the first column). I´m trying to do SVR-RBF.

I did 10fold , and gotta better results with gamma 8.

But, when apllied to my external test set, I gotta bad results (overfit?).

So, I tried to minimize gamma, and get good results (in the test set) with 0.0625.

How this is possible ?

Why I´m getting good cross validation results, and bad test results ?

Why I´m getting worse cross validation results, and better test results ?

Thanks in advance.

Andre

$\endgroup$

1 Answer 1

1
$\begingroup$

High values of $\gamma$ (low bandwidth) induce more complex models. It seems that you have a classical case of overfitting on the training set. However, cross-validation should pick up on this as well, since it is intended to provide good estimates of generalization performance.

Cross-validation works assuming the test set is similar to the training set. Is this the case in your application or is it possible that your test set is fundamentally different?

$\endgroup$
1
  • $\begingroup$ Marc, both training and test sets are artificially done by an simulator. What I did was, for the training, simulation of targets 1 and 5, so, for the test set, Ive simulated 3 for example. The data has the same fundamental origins. $\endgroup$ Nov 11, 2014 at 15:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.