How to interpret the model parameters of libsvm via MATLAB interface?

Question

I used the MATLAB interface of libsvm for doing binary classification of 997-dimensional training data. I am trying to understand how the resulting model is used to compute the predicted output (which we get by calling svmpredict)

The model contains fields (it has linear kernel):

nSV = [546; 246]; totalSV=792; rho = 0.093
and svCoeff [792x1 double] and SVs [792x997 double]

I thought that we must be simply multiplying svCoeff with SVs to get a [997x1] matrix which we then multiply with the actual feature, before shareholding by rho. But that's not the case. Can someone illustrate with a simple equation how these parameters are used to do classification?

Answer 1

Support vector machine classifiers use the following decision function to determine the label for a test instance $\mathbf{z}$:

$f(\mathbf{z})=\mathtt{sign}\big(\sum_{i=1}^{totalSV} y_i \alpha_i \kappa(\mathbf{x}_i,\mathbf{z})-\rho\big)=\mathtt{sign}\big(\langle\mathbf{w},\Phi(\mathbf{z})\rangle-\rho\big)$,

where $\kappa(\cdot,\cdot)$ is the kernel function, $\alpha$ contains the support values, $\mathbf{y}$ is the training label vector, $\rho$ is a bias term and $\mathbf{w}$ is the separating hyperplane in feature space.

In a LIBSVM model, sv_coef contains $\alpha_i y_i$ and SVs contains the support vectors ($\mathbf{x}_i$). To predict you need to perform kernel evaluations between the test point and all support vectors.

For the linear kernel ($\kappa(\mathbf{x},\mathbf{z})=\mathbf{x}^T\mathbf{z}$) you can compute $\mathbf{w}$ explicitly:

$\mathbf{w}=\sum_{i=1}^{totalSV} \alpha_i y_i \mathbf{x}_i=\mathtt{sv\_coef}^T \times \mathtt{SVs}$.

Subsequently, predictions are simply based on the sign of $\mathbf{w}^T\mathbf{z}-\rho$.

Answer 2

Nevermind, I found svm.cpp in the svmlight package and read the svm_predict function. It is written for the general case for n classes but for the simple case of two classes their logic boild down to

>> sv=model.SVs;
>> svc=model.sv_coef;
>> sv546=sv(1:546, :); %Since model.label is [1, -1] and model.nSV=[546; 246]
>> sv246=sv(547:end, :);
>> svc546=svc(1:546);
>> svc246=svc(547:end);
>> weight_for_minus1=transpose(svc246)*sv246; %Since model.label is [1, -1] and model.nSV=[546; 246]
>> weight_for_plus1=transpose(svc546)*sv546;
>> 'now multiply weight_for_minu1 and weight_for_plus1 with the 997-dimensional feature and select whichever is positive'