Combining time series anomaly measures

Question

suppose I have different "scores" that can be computed for every data point in a time series. The score quantifies how anomalous the point is. The scores are very different in nature, so its not like an ensemble of identical algorithms on different features.....

I want to understand what possibilities I have to combine them and find it difficult to get an overview. Approaches I have already thought of:

• Thresholding of every single score to get a classifier and then combine them via a voting approach (majority decision if anomaly or not)

• Average of scores and then threshold the average to have one decision

For these approaches, there is also the question how to get a weighting for the different scores.

It would be great if you can point me into some direction on what methods are out there to approach something like this.

Thank you!

Edit: I try to give a little more context to make it easier understandable:

We want to look at every data point of a time series and return information regarding if it is an anomaly (either by yes/no or a score between 0 and 1).

We have several "classfiers" that return this output:

• Based on some domain knowledge, a threshold is defined and every point above is an anomaly with some probability

• A machine learning algorithm returns the probability/decision

• Decision/probability is based on mesauring similarity with some reference time series

• .....

I am looking for ideas to combine the scores or decisions into a final one. Maybe this is still to general, i am just trying to get an idea on what is out there.

Thank you.

Answer 1

If you have access to the "ground truth" for at least part of your time series, i.e., knowledge whether a given data point is anomalous or not, then I would recommend one of two possibilities.

1. The first approach is a two-step procedure:

   1. First, turn your scores into probabilistic predictions of whether a given point is anomalous or not. For instance, you can simply run a logistic regression of your ground truth on the scores, separately for each score. If you have enough data and suspect nonlinearities, it can be worthwhile to transform the scores using splines.

      Applying these models to future scores yields probabilistic predictions for future time points to be anomalous. The advantage is that these probabilistic predictions are comparable.

   2. You can now simply take an unweighted average of the predictions coming out of each score (and its associated separate model). You could also try weighted combinations, but these are often outperformed by simple unweighted combinations (the "forecast combination puzzle").

2. Alternatively, simply run a single logistic regression of your ground truth on all the scores. Or any other probabilistic classifier model - a GLMNet often works very well. Applying this model to future scores will again give probablistic predictions for a future time point to be anomalous.

In either case, you can now work with these probabilistic classifications directly, or, if you understand your decision framework, use one or multiple thresholds to turn your probabilistic predictions into actions.