0
$\begingroup$

Im trying to create a one class SVM using libSVM. However whenever I run the svmpredict function it always return an accuracy of 0%.

model = svmtrain(label,userProfile',('-s 2 -t 2 -c 1 -g .8'))
*
optimization finished, #iter = 6
obj = 4.792359, rho = 2.225069
nSV = 6, nBSV = 3

model = 

    Parameters: [5x1 double]
      nr_class: 2
       totalSV: 6
           rho: 2.2251
         Label: []
    sv_indices: [6x1 double]
         ProbA: []
         ProbB: []
           nSV: []
       sv_coef: [6x1 double]
           SVs: [6x30 double]

>> svmpredict(1,x2',model)
Accuracy = 0% (0/1) (classification)

ans =

    -1

Might it have something to do with my data preparation? Can anybody suggest what I might be doing wrong?

$\endgroup$
  • 1
    $\begingroup$ With only one test point you are bound to get extreme results ($100\%$ or $0\%$ accuracy). Do you see the same with a reasonable test set? $\endgroup$ – Marc Claesen Jan 22 '14 at 22:14
  • $\begingroup$ Ill only ever plan to test one point at a time, its for a bio-metric authentication system. I'm new to stats and machine learnng, is there a better way I should be going about this? $\endgroup$ – Steve2056726 Jan 22 '14 at 22:26
  • $\begingroup$ To assess quality of your model you need to test on a large set of data. That's the case even if later on you want to deploy and test one point at a time. I would suggest reading some introductory material on machine learning before going any further... $\endgroup$ – iliasfl Jan 23 '14 at 5:35
  • $\begingroup$ I've done an on-line course with Adnrew Ng and read a few papers regarding my problem space. The problem I'm trying to address is the ability to authenticate users based on their keystroke timing patterns during login. Therefore I cant get testing data of imposters trying hack into the account for security reasons. Would it possible to generate data for the testing purposes? $\endgroup$ – Steve2056726 Jan 23 '14 at 11:40
1
$\begingroup$

OK, interesting problem. Let's say that you manage to built the perfect model: using one-class SVM you model where in feature space the legitimate users live.

How do you plan to assess it and prove that it works? At the end of the day you can't avoid having a test set. If you can't find in literature a model for simulating imposters or some publicly available dataset, unfortunately you can't do much.

So, my advice is this: you can at least try to build your own dataset. Physicists, biologists, even social scientists build their own datasets all the time. From my experience, it is usually mathematicians/statisticians that keep on believing that generating data for solving a problem or validating a solution is somebody else's job.

$\endgroup$
  • $\begingroup$ Good point however I imagine that OP would be also happy to know how to build such dataset since he does not have one. $\endgroup$ – Tim Apr 27 '15 at 9:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.