3
$\begingroup$

I'm trying to mess around with a one-class SVM implementation I hacked together from ArduinoSVM.

I'm using an RBF kernel and training the model with just "in" datapoints with sklearn.

First, as is good practice, I scale the "in" datapoints (between -1 and 1) before I put them in the SVM. After playing around, trying random numbers in the prediction function on a microcontroller, it just spits out a result of 0.000000000000 where it should return a negative value, meaning the random numbers are classified as "out" of the dataset, or novelties.

This is, as far as I can gather, because the rbf kernel returns exp(-gamma * result) where result is a function of one of the datapoints minus a constant. The issue is that I'm putting random numbers in that are way above/below the previously seen "in" data that is used to inform the scaling algorithm.

As such the scaling algorithm is turning my "2000" into "70000000" rather than a number between -1 and 1. Then trying to raise e to the power of it, even with double floating point math in a microcontroller - that's not going to work! (because it's too small to represent!)

So - this comes to my question, what's the best way to circumvent this? Off the top of my head, here are my options:

  1. Ideally you have a training dataset with "out" or novelty data so the scaling algorithm can be generated accordingly, but this counters the whole idea of novelty detection!
  2. In my microcontroller code, look for a really high (or low) number away from 1 and -1 and say it's not in the dataset before I even predict with it. But isn't this what the SVM is supposed to be doing?
  3. I get that the number returned is negative, it's just too negative to be represented as such, I could just check in my code whether the result is <= 0. Which would catch this case...

I look forward to hearing anyone's thoughts on this please! I've put a more comprehensive explanation of the problem here https://workyourtech.com/2020/04/11/svm-scaling-issues-with-one-class-novelty-detection/

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.