Outlier/Anomaly Detection Algorithm Options

Question

I am trying to determine the best algorithm/approach to monitor an application at my company.

The simple overview is that when our customers call in to report an issue, we run automated diagnostics to help isolate the issue. The automation returns a number of different codes back to represent the results. IE, a 1 may be a "No Issue Detected", a 2 may be "Server has high CPU", and so on. There are 50 or so different ones, but only 5-10 that have much volume.

What I am trying to do is use anomaly detection to monitor this automation. Sometimes the automation continues to work, but some of the sub-components of the tests start to fail and is not noticed in a timely manor.

The volume of total tests are dependent on the number of troubles reported by the customer. It is generally fairly predictable, but events like major weather events etc cause spikes. There is also a seasonal component. I have about 500 days of data, though due to improvements in the automation, etc, all of it is probably not very useful.

I create a time series by day of each results total volume. So, Day 1 - 50, Day 2- 40, Day 3-37, etc etc and train the model.

I have tried using Isolation Forest and Prophet, but both have issues.

One primary issue is that for low volume results, the range of of values allows for negatives. Even when I use the floor/cap in prophet. And even that has other problems, as I know the floor(zero), but the cap is not as easily determined.

In my scenario the lowest possible value is obviously zero. So, if one of the result codes ends up being zero on one day, that could trigger as an anomaly, but since it generally has between 10-20, that ends up within the y_hat_lower range(in prophet). And usually that value even dips into negative ranges.

I tried converting each result code to a percentage of the total volume, and I still get this issue. Basically, negative values should not be allowed in this scenario. Because they are not possible. You would think a model that is trained on 400 days of data with the lowest value being zero and the majority being above zero would pick up on that.

I'm trying to determine if a different model(SVM/LOF/ARIMA) would suit my dataset better.

I get why and how these ranges are generated on forecasted type data. It makes sense, if there is a downward trend on something that has low volume it would generally assume the value would go negative, but at the same type, there has to be models that understands that zero is an absolute low as well.

Any suggested reading, etc. would be greatly appreciated.

Answer 1

To be strict, your data is a time series of count data and thus cannot be handled by most of the standard models like ARIMA. However, it is very often done nonetheless and then considered as an approximation. Yet, there are libraries that provide models for time series of count data, e.g. the R package tscount. It provides forecasting which could be used for outlier detection.

Another approach that I would suggest is transforming your time series $y_t$ into: $$ z_t := \log(y_t+s), $$ where $s$ is some small number, e.g. $0.1$, added only to make sure that your data never becomes zero. Then you do ordinary time series analysis with this new time series which doesn't have any constraints anymore. This has the advantage of being straightforward and you can immediately apply standard libraries for detecting outliers in time series as e.g. tsoutliers.

Last but not least, there are also more sophisticated ways of analyzing positive time series in general, like e.g. PTSR. However, as far as I can see, here you have to do the outlier detection part of the task yourself.