suppose I have different "scores" that can be computed for every data point in a time series. The score quantifies how anomalous the point is. The scores are very different in nature, so its not like an ensemble of identical algorithms on different features.....

I want to understand what possibilities I have to combine them and find it difficult to get an overview. Approaches I have already thought of:

  • Thresholding of every single score to get a classifier and then combine them via a voting approach (majority decision if anomaly or not)

  • Average of scores and then threshold the average to have one decision

For these approaches, there is also the question how to get a weighting for the different scores.

It would be great if you can point me into some direction on what methods are out there to approach something like this.

Thank you!

Edit: I try to give a little more context to make it easier understandable:

We want to look at every data point of a time series and return information regarding if it is an anomaly (either by yes/no or a score between 0 and 1).

We have several "classfiers" that return this output:

  • Based on some domain knowledge, a threshold is defined and every point above is an anomaly with some probability

  • A machine learning algorithm returns the probability/decision

  • Decision/probability is based on mesauring similarity with some reference time series

  • .....

I am looking for ideas to combine the scores or decisions into a final one. Maybe this is still to general, i am just trying to get an idea on what is out there.

Thank you.

  • $\begingroup$ Do you have a ground truth of data points explicitly labeled as anomalous? $\endgroup$ Commented Jul 24, 2022 at 19:20
  • $\begingroup$ Hi. Part of the data can be seen as normal (not containing anomalies) and can be used for training/calibration/finding thesholds. But the whole problem is rather unsupervised/semi-supervised. $\endgroup$ Commented Jul 26, 2022 at 16:11
  • $\begingroup$ Hm. I would not try to train a classifier on "normal" data alone. I think if you gave more context, it might be easier to help you. $\endgroup$ Commented Jul 26, 2022 at 17:19
  • $\begingroup$ Thanks for your response, I tried to add more info in the original post. $\endgroup$ Commented Jul 30, 2022 at 12:47
  • $\begingroup$ The problem still is that if you don't have the ground truth for at least some of your data, it seems like a rather fruitless exercise. How would you know you are doing the right thing? You could "combine" your scores by labeling all data points as "anomalous", or none of them, or just pick one of your classifiers at random. How would you know any of these methods don't work? $\endgroup$ Commented Jul 30, 2022 at 12:50

1 Answer 1


If you have access to the "ground truth" for at least part of your time series, i.e., knowledge whether a given data point is anomalous or not, then I would recommend one of two possibilities.

  1. The first approach is a two-step procedure:

    1. First, turn your scores into probabilistic predictions of whether a given point is anomalous or not. For instance, you can simply run a logistic regression of your ground truth on the scores, separately for each score. If you have enough data and suspect nonlinearities, it can be worthwhile to transform the scores using .

      Applying these models to future scores yields probabilistic predictions for future time points to be anomalous. The advantage is that these probabilistic predictions are comparable.

    2. You can now simply take an unweighted average of the predictions coming out of each score (and its associated separate model). You could also try weighted combinations, but these are often outperformed by simple unweighted combinations (the "forecast combination puzzle").

  2. Alternatively, simply run a single logistic regression of your ground truth on all the scores. Or any other probabilistic classifier model - a GLMNet often works very well. Applying this model to future scores will again give probablistic predictions for a future time point to be anomalous.

In either case, you can now work with these probabilistic classifications directly, or, if you understand your decision framework, use one or multiple thresholds to turn your probabilistic predictions into actions.

  • $\begingroup$ Thank you very much for your answer. Regarding 1.1 : For the logisitc regression, I assume that the y is the ground truth, the x is the anomaly score and the beta (regression coefficients) are fitted during training? $\endgroup$ Commented Jul 31, 2022 at 17:57
  • $\begingroup$ Yes, exactly. (Betas only come in if you run a logistic regression or similar. You could also use some black box ML classifier, as long as it returns probabilistic classifications.) $\endgroup$ Commented Jul 31, 2022 at 18:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.