1
$\begingroup$

For a classification problem I am giving the R package kernlab a shot – not the least because it offers to calculate class probabilities instead of only a plain decision. However, comparing results for both options appears to be contradictory. Some code for a 4 elemented testing data set:

model <- ksvm(x=formula,data=data[trainIndex,],
  prob.model=TRUE, type="C-svc");
predProb <- predict(model,data[testIndex,],type="probabilities");
predClass <- predict(model,data[testIndex,]);

In my 4 class case this led to:

> predProb
    desmino   filamino  myotilino     titino
[1,] 0.06676130 0.03833511 0.08397281 0.81093078
[2,] 0.06832143 0.83437244 0.03403721 0.06326892
[3,] 0.03655672 0.02404717 0.05114122 0.88825489
[4,] 0.03305530 0.05722160 0.68772201 0.22200108
> predClass
[1] myotilino filamino  titino    myotilino
Levels: desmino filamino myotilino titino

I am disturbed by the first line in the matrix. Clearly the probability attached to titino accounts for for the highest value. Still myotilino won in terms of predicted class. Is this a bug, or is there something I did not understand?

Improve this question
$\endgroup$
3
  • $\begingroup$ Looks like a bug to me. $\endgroup$
    – Marc Claesen
    Commented Apr 1, 2014 at 8:13
  • $\begingroup$ This of course is only an example where I omitted the data (it is a large 38x664 object). For completenesses sake: The true value would be 'titino'. $\endgroup$
    – Markus-Hermann
    Commented Apr 1, 2014 at 8:18
  • $\begingroup$ From another source it has been suggested, that the dataset may be too small (which in my humble opinion still would not warrant a contradiction in results). So here goes some more information: The training set contains 34 data points, the testing set the 4 points used in the example. For the 663 possible predictors it is guaranteed, that each column has 80% values >0 which are spectral counts from a proteomics experiment. Using either the svms from e1071 or ksvm I usually get accuracy values (i.e. the rate of correct predictions) of roughly 80% when doing a 10-fold cross validation. $\endgroup$
    – Markus-Hermann
    Commented Apr 1, 2014 at 13:26

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.