I'm trying to mess around with a one-class SVM implementation I hacked together from ArduinoSVM.
I'm using an RBF kernel and training the model with just "in" datapoints with sklearn.
First, as is good practice, I scale the "in" datapoints (between -1 and 1) before I put them in the SVM. After playing around, trying random numbers in the prediction function on a microcontroller, it just spits out a result of 0.000000000000 where it should return a negative value, meaning the random numbers are classified as "out" of the dataset, or novelties.
This is, as far as I can gather, because the rbf kernel returns exp(-gamma * result) where result is a function of one of the datapoints minus a constant. The issue is that I'm putting random numbers in that are way above/below the previously seen "in" data that is used to inform the scaling algorithm.
As such the scaling algorithm is turning my "2000" into "70000000" rather than a number between -1 and 1. Then trying to raise e to the power of it, even with double floating point math in a microcontroller - that's not going to work! (because it's too small to represent!)
So - this comes to my question, what's the best way to circumvent this? Off the top of my head, here are my options:
- Ideally you have a training dataset with "out" or novelty data so the scaling algorithm can be generated accordingly, but this counters the whole idea of novelty detection!
- In my microcontroller code, look for a really high (or low) number away from 1 and -1 and say it's not in the dataset before I even predict with it. But isn't this what the SVM is supposed to be doing?
- I get that the number returned is negative, it's just too negative to be represented as such, I could just check in my code whether the result is <= 0. Which would catch this case...
I look forward to hearing anyone's thoughts on this please! I've put a more comprehensive explanation of the problem here https://workyourtech.com/2020/04/11/svm-scaling-issues-with-one-class-novelty-detection/