I have a small dataset (20 instances per 13 classes). The 13 classes are human users from their behavior features, I have to classify if an unseen behavior feature is of a user or not.

Data: These features are basically mouse events (x, y, timestamp) in a sequence upto 200 elements-long. And there is also an associated bi-gram key press time which has been binned into a range of 200 bins with 1 ms width. A user would have 20 events as [[[x, y, time]...], [0,0,0,..,1,2,0,3,..]]

Problem: I have tried several statistical techniques, and of all, SVMs with linear kernel are giving the best result, but the result is far from decent. I have trained an SVM per user, and the average false acceptance rate is only 25%, with false rejection rate around 7%.

I am not a seasoned data scientist and just breaking into the field, what are the ways I can go around improving the solution? (In terms of algorithms, technqiues to try) I have already tried K-means, Distance-based, random forests. Is my data too small to even capture the hypothesis? Can methods like one-shot learning be applied.

Background: I tried to create a heatmap of the mouse-movements, and there does seem to be patterns tied to users. Here are two plots for different users.

enter image description here

enter image description here

Here's a sample of the dataset


As you mention, your dataset is quite small. Personally I think that it is too small for exploring the implementation of a model, but trying a few things wouldn't hurt.

There is a python package called smote which helps in oversampling/undersampling. This method is mainly used when dealing with imbalanced datasets , but you could try implementing it and creating new artificial entries. You can check out the docs here: SMOTE and the paper, on which it is based here: SMOTE paper . Please bare in mind that by following this method and creating artificial instances you might alter the dataset so much that the initial information is lost thus resulting to a model which will not address your task properly. Also, a synthetic dataset means that you will probably have to deal with overfitting.

Moreover, since you have tried multiple models, you could combine them (model ensembling) and create a multiple model voting system. Sklearn has a method for this from the ensemble class Voting Classifier


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.