Is there a right way to aggregate timeseries data for anomaly detection and forecasting?

Question

Hello Cross Validated Users,

I have had a question on my mind for a long time whenever I work with timeseries but I have never quite found the right answer.

Here is the thing: when creating an timeseries forecasting model with the goal of doing anomaly detection, most of the time the creation of the timeseries is done by aggregating the data by dates and count the number of occurences, or sum/mean/med a value relevant to the use case for each timestep.

However, most of the time it is possible to aggregate the data according to different combinations of features. For example, one could aggregate by [Name, City] or by [Name, City, Birthdate], and have a timeseries for each "subset".

Different subsets of features gives different cardinality, with some of them being relevant for the anomaly detection phase as they can have more variance with peak anomaly (that would not be seen when aggregating with a lower-order cardinality subset).

The question is : is there a right way to approach the processing phase ? Should one just train a model to predict each subset as a different timeseries and introduce a metric that average the anomaly score after the forecasting ? Or is there a way to aggregate all the differents subsets into one timeserie, while keeping most of the information (variance, deviation, anomalies...) ?

Thank you in advance if some of you have any inputs.

Answer 1

I think this depends on what decisions you make with the anomaly information. I guess your detection must be followed by some action or intervention. What is important for the business? I think if you make it too granular you have too many anomalies and the data might be quite sparse.

Maybe you start at the high level and then work your way downwards to detect which subset "caused" the high level anomaly.