3
$\begingroup$

I am using SVM-light with Matlab, for linear SVM. I would like to understand the output model, but I cannot find any documentation or help about it.

Here is the output:

        sv_num: 639
   upper_bound: 547
             b: 1.4023
      totwords: 576
        totdoc: 2000
     loo_error: -1
    loo_recall: -1
 loo_precision: -1
      xa_error: 14.9500
     xa_recall: 82.6000
  xa_precision: 86.8559
       maxdiff: 9.9611e-004
    r_delta_sq: 575.0052
   r_delta_avg: 23.9792
  model_length: 0.4847
          loss: 357.5542
         vcdim: 136.1071
         alpha: [2000x1 double]
   lin_weights: []
         index: [2000x1 double]
        supvec: [639x576 double]
   kernel_parm: [1x1 struct]
example_length: 23.9792
             a: [2000x1 double]

So far, I understand there are sv_num=639 support vectors. However, there are 638 positive index in the index vector ; ditto for the positive values of the alpha vector. The first value of the alpha vector is zero, and the non-zero values are then between 2 and 639. Same remark about the supvec vector. So it seems there are indeed 638 support vectors.

Actually, I would like to get the hyperplane W and the parameter b myself to see if I could understand the output model correctly. Since the kernel is linear, I thought of a sum over some coefficients times some support vectors.

For information, a(i) "should" store the values alpha_i times y_i for training samples x_i. And there are 638 non-zero values.

Has anyone used svmlight before? Can you understand the output model? Or is there a documentation anywhere?


I manage to get one thing right:

MyCoeff = model.a ./ y;
MyIndex = 1+max(model.index,0); % between 1 and 639

The support vector corresponding to any non-zero MyCoeff(i) is:

  • x(i,:)
  • model.supvec(MyIndex(i),:)

And MyIndex is 1 iff MyCoeff is zero. And supvec(1,:) is the zero vector.


Here are four pieces of code I have made:

MyCoeff = model.a ./ y;
W1 = zeros(1, h*w);
for i=1:length(MyCoeff)
    W1 = W1 + MyCoeff(i)*x(i,:);
end;

W2 = zeros(1, h*w);
for i=1:model.sv_num
    W2 = W2 + model.alpha(i)*model.supvec(i,:);
end;

eq((MyCoeff~=0),(model.index>0))
MesIndices = 1+max(model.index,0); % entre 1 et 639
W3 = zeros(1, h*w);
for i=1:length(MyCoeff)
    W3 = W3 + MyCoeff(i)*model.supvec(MesIndices(i),:);
end;

W4 = zeros(1, h*w);
for i=1:length(MyCoeff)
    W4 = W4 +  model.alpha(MesIndices(i))*x(i,:);
end;

Here are the outputs: http://imgur.com/FErWW.png

I have understood the index link between the support vectors supvec and the matrix x. But I have not found the link between alpha and a.


I got it: apart from the difference of indices, model.a and model.alpha are the same.

for i=1:length(MesIndices)
    W1 = W1 + model.a(i)*x(i,:);%*y(i);
    W2 = W2 + model.alpha(MesIndices(i))*model.supvec(MesIndices(i),:);%*y(i);
    W3 = W3 + model.a(i)*model.supvec(MesIndices(i),:);%*y(i);
    W4 = W4 +  model.alpha(MesIndices(i))*x(i,:);%*y(i);
end;

The question now is whether I should multiply by y(i). I believe I should NOT since model.a is already y times the weight of the corresponding support vector, and $y\in\{-1,1\}$. So the good result should be the ORANGE face, which you can get by using one of the four lines above. Too bad I could not find any documentation. :')

$\endgroup$

1 Answer 1

0
$\begingroup$
sv_num: one added to the number of support vectors
     b: offset
 alpha: 0, then "a" corresponding to the support vectors, then only zeros
 index: link index between "a" and alpha, "x" and supvec 
supvec: 0, then "x" corresponding to the support vectors
     a: weights_i (between 0 and C) times y_i
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.