I have some time series data I'm working with, where each at each step I have a sequence. For example I could have a bunch of graphs looking like the left figure, followed by an anomalous piece of data looking like the right figure.

enter image description here

I want to construct an online method where I'm able to say the anomalous graph is clearly from a different distribution or similar approach which only uses the recent data, say the previous 20 graphs. Is there a nice way to do this or some nice literature for this type of problem? I've tried some optimal transport methods to compare distributions and simple stats, but I want something that is fairly robust. Any comments welcome!

The anomalies can be anything - for instance the right-hand image is showing jagged behaviour rather than smooth, but generally the anomalies are not visually subtle, the distributions will look a lot different.

  • 2
    $\begingroup$ What kind of anomalies do you have in mind (it's not clear for me from the image)? $\endgroup$
    – Tim
    Apr 30 '18 at 12:11
  • $\begingroup$ Please register &/or merge your accounts (you can find information on how to do this in the My Account section of our help center), then you will be able to edit & comment on your own question. $\endgroup$ Apr 30 '18 at 13:00

There is no general solution to the anomaly problem, but you can address specific cases. For example.

  • Are the "previous 20" all trusted cases (i.e. not anomalies)? If they are short pieces of time series all of equal length, you could do a functional PCA, hopefully work with one or two principal components, and then assess if your new curve departs significantly from the others on the basis of the first and/or second PC scores.
  • Are you primarily distinguishing smooth functions from jagged ones? I recently used double exponential smoothing in a case like this. With smooth data, exponential smoothing will optimize with a beta value close to 1 and an alpha close to 0 (it basically tries to grab first differences of the data as it attempts to estimate the derivative of the function). When you apply those parameter values to a jagged portion of the series, your forecast values will depart markedly from your actual values -- so the sequence of anomalies you get will be an indication that you have departed smooth territory.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.