I am considering repeated time series (for example on day 1, day 2, ..., day(n)), and I want to find the anomalous points in the day(n+1) given that some delay between samples across different days is allowed.

To provide an explanation on my objective, let's consider the imaginary plot shown below, in which I have three time series (on day 1, day 2, and day 3). Let's say I already have the knowledge that there is no anomaly in these time series and all of them have a normal trend. As we can see, there is also some time delay between the sharp peaks across different days and they don't occur at the same time. However, they are not considered as anomaly because the trend (time-series signature or shape) is preserved across all three time series.

Figure 1

Now, let's consider I add the 4-th time series, and I want to find the anomalous points in this one (based on the previous observations, I have an estimation of normal trend). As shown, there are three zones in the 4-th time series that are anomalous since the trend in these zones deviate a lot from those of the previous days.

figure 2

My question is, what is a suitable algorithm for detecting these anomalous regions? The characteristics of my dataset is that (1) I have several observations which I know are normal; however, there is some uncertainty in terms of having time delay between peaks in normal observations, and the peaks are not necessarily located at the same time on different days and (2) I'm interested to detect anomalous zones (a sample of subsequent points that represent a signature) rather than focusing on a single point.

I already tested two simple models but got many false positives in areas that have peaks: (1) Subtracting the 4-th time series from the average of the previous ones. The point is considered anomalous if the difference (diff) is outside the range of mu(diff) +- 3*std(diff), (2) developing a prediction model (like Gaussian Process Regression) on the 3 normal observations , considering a prediction interval (like 99%), and then checking if samples in the new observation are outside the prediction interval.

  • $\begingroup$ why is machine-learning tag? $\endgroup$ – Aksakal Apr 9 '18 at 18:48
  • $\begingroup$ Reconstructing the time series after performing SVD and comparing the timeseries will give you anomaly scores for all the points. Just use a threshold to get the largest scores. $\endgroup$ – kedar tamboli Apr 10 '18 at 11:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.