Can you use the isolation forest algorithm on a large sample size?

Question

I've been using the scikit learn sklearn.ensemble.IsolationForest implementation of the isolation forest to detect anomalies in my datasets that range from 100s of rows to millions of rows worth of data. It seems to be working well and I've overridden the max_samples to a very large integer to handle some of my larger datasets (essentially not using sub-sampling). I noticed that the original paper (https://ieeexplore.ieee.org/abstract/document/4781136) states that larger sample sizes create risk of swamping and masking.

Is it okay to use the isolation forest on large sample sizes if it seems to be working okay? I tried training with a smaller max_samples and the testing produced too many anomalies. My data has really started to grow and I'm wondering if a different anomaly detection algorithm would be better for such a large sample size.

Answer 1

In my scenario, I am also running IForest from sklearn on a couple of million rows without any troubles. However I am running 1000 n_estimators with a max_samples of 10.000 so nothing huge in these regards, but I did not notice any accuracy improvements by increasing these numbers other than just a performance hit.

I had the same question you do, whether another algorithm is beneficial, but when I tried running a OneClassSVM on my dataset it took forever to train.

So I would just stick with isolation forests as I at least have experienced they perform quite well on large datasets.