0
$\begingroup$

I run epsilon-svr on about 1000 samples (RBF kernel), and then calculate the in-sample training errors.

The hyperparameters are found via a search, ranging from 1e-5 to 1e2. model->eps is a couple orders of magnitude lesser than model->p.

From my understanding, the KKT conditions are:

1. sv_coef[i] == 0         <==>   abs( predict(prob->x[i]) - y[i] )  <= model->param.p,
2. 0 < abs(sv_coef[i]) < C <==>   abs( predict(prob->x[i]) - y[i] )  == model->param.p  (or extremely close)
3. abs(sv_coef[i]) == C    <==>   abs( predict(prob->x[i]) - y[i] ) >= model->param.p
4. abs(sv_coef[i]) <= C

Conditions 1,2,3 break for me over many of the training vectors, e.g.:

Violated KKT conditions for RemainingSet: Weight/Cost: 0/700, label: 1.00288, predicted: 1.0025, Error/Eps: 0.000372128/0.0001, computed violation: 0.000272128

As printed, a non-support vector does not like inside the epsilon tube. The algorithm does not hit max iterations, so it claims that it has converged to the global optimum.

Does LibSVM output a broken system in epsilon-SVR mode? Is the error caused by numerical stability? How would one go about to stabilize an SVM with violated KKT conditions?

$\endgroup$
0
$\begingroup$

LibSVM uses 1-indexed counting for a certain structure, which must be dealt with - the svm_model.sv_indices array. When used correctly, all 4 KKT conditions hold in my case.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.