I am using SVM-light with Matlab, for linear SVM. I would like to understand the output model, but I cannot find any documentation or help about it.
Here is the output:
sv_num: 639
upper_bound: 547
b: 1.4023
totwords: 576
totdoc: 2000
loo_error: -1
loo_recall: -1
loo_precision: -1
xa_error: 14.9500
xa_recall: 82.6000
xa_precision: 86.8559
maxdiff: 9.9611e-004
r_delta_sq: 575.0052
r_delta_avg: 23.9792
model_length: 0.4847
loss: 357.5542
vcdim: 136.1071
alpha: [2000x1 double]
lin_weights: []
index: [2000x1 double]
supvec: [639x576 double]
kernel_parm: [1x1 struct]
example_length: 23.9792
a: [2000x1 double]
So far, I understand there are sv_num=639
support vectors. However, there are 638 positive index in the index
vector ; ditto for the positive values of the alpha
vector. The first value of the alpha
vector is zero, and the non-zero values are then between 2 and 639. Same remark about the supvec
vector. So it seems there are indeed 638 support vectors.
Actually, I would like to get the hyperplane W
and the parameter b
myself to see if I could understand the output model correctly. Since the kernel is linear, I thought of a sum over some coefficients times some support vectors.
For information, a(i)
"should" store the values alpha_i
times y_i
for training samples x_i
. And there are 638 non-zero values.
Has anyone used svmlight before? Can you understand the output model? Or is there a documentation anywhere?
I manage to get one thing right:
MyCoeff = model.a ./ y;
MyIndex = 1+max(model.index,0); % between 1 and 639
The support vector corresponding to any non-zero MyCoeff(i)
is:
x(i,:)
model.supvec(MyIndex(i),:)
And MyIndex
is 1 iff MyCoeff
is zero. And supvec(1,:)
is the zero vector.
Here are four pieces of code I have made:
MyCoeff = model.a ./ y;
W1 = zeros(1, h*w);
for i=1:length(MyCoeff)
W1 = W1 + MyCoeff(i)*x(i,:);
end;
W2 = zeros(1, h*w);
for i=1:model.sv_num
W2 = W2 + model.alpha(i)*model.supvec(i,:);
end;
eq((MyCoeff~=0),(model.index>0))
MesIndices = 1+max(model.index,0); % entre 1 et 639
W3 = zeros(1, h*w);
for i=1:length(MyCoeff)
W3 = W3 + MyCoeff(i)*model.supvec(MesIndices(i),:);
end;
W4 = zeros(1, h*w);
for i=1:length(MyCoeff)
W4 = W4 + model.alpha(MesIndices(i))*x(i,:);
end;
Here are the outputs:
I have understood the index
link between the support vectors supvec
and the matrix x
. But I have not found the link between alpha
and a
.
I got it: apart from the difference of indices, model.a
and model.alpha
are the same.
for i=1:length(MesIndices)
W1 = W1 + model.a(i)*x(i,:);%*y(i);
W2 = W2 + model.alpha(MesIndices(i))*model.supvec(MesIndices(i),:);%*y(i);
W3 = W3 + model.a(i)*model.supvec(MesIndices(i),:);%*y(i);
W4 = W4 + model.alpha(MesIndices(i))*x(i,:);%*y(i);
end;
The question now is whether I should multiply by y(i)
. I believe I should NOT since model.a
is already y
times the weight of the corresponding support vector, and $y\in\{-1,1\}$. So the good result should be the ORANGE face, which you can get by using one of the four lines above. Too bad I could not find any documentation. :')