I have huge multivariate time series to analyze (Terabytes of data) and I need fast, scalable algorithms for mainly two tasks:

  • finding similar patterns among time series. For example, imagine I identify a certain pattern in a reference time series. I have a group of different time series, and I want to know in which of them I can find a similar pattern (let's say, a similar shape), and in which part of the time series. In the past, I used to use Dynamic Time Warping as a measure of similarity among time series (to be more precise, as a metric), and I liked it a lot because it's an extremely intuitive method. However, DTW is not precisely what I would call a "scalable" algorithm (but I may be wrong, in which case please correct me). Instead, I've been told I should switch to SAX.

  • anomaly detection for multivariate time series. For example, I acquire multiple sensor measurements from a machine in "normal operation". Then I acquire further data for the same machine. I need to identify when the behavior of the machine differs significantly from what I considered "normal operation".

Having a look at Mueen and Keogh's tutorial on the Matrix Profile, I thought it could be of help (or at least time series motifs may be of help) for my goals. However, I'm not familiar with this kind of algorithms for time series analysis (I'm not an expert of time series in general). Thus, before embarking on a time-consuming literature study, I'd like to know:

  • am I right? Are these tools useful for my goals, or are there better approaches?
  • can you give me the intuition about these tools? Of course I don't expect you to show me a detailed implementation, but something which could help me digest faster the relevant literature.

Yes, the Matrix Profile allows discord discovery, which is very competitive for anomaly detection (according to multiple independent test)

And yes, while "finding similarities among time series" is a bit too vague to clearly respond to, the Matrix Profile does do that.

If you write to the author of the tutorial (me) with some data samples, he will advise more.

  • $\begingroup$ thanks! Are you the first author or the second one? I'll write to both, just in case :-) Concerning "finding similarities among time series", I will modify that part of the question so that it makes more sense. Let me know if it's an improvement. I mean, I could also just contact you directly, but I think it's fair for the Community here to get a better question. $\endgroup$ – DeltaIV Mar 23 '18 at 17:08
  • $\begingroup$ ok, I got it, you're the second author :-) $\endgroup$ – DeltaIV Mar 23 '18 at 17:17
  • 1
    $\begingroup$ I am the good looking author ;-) $\endgroup$ – user2313186 Mar 24 '18 at 4:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.