I have huge multivariate time series to analyze (Terabytes of data) and I need fast, scalable algorithms for mainly two tasks:
finding similar patterns among time series. For example, imagine I identify a certain pattern in a reference time series. I have a group of different time series, and I want to know in which of them I can find a similar pattern (let's say, a similar shape), and in which part of the time series. In the past, I used to use Dynamic Time Warping as a measure of similarity among time series (to be more precise, as a metric), and I liked it a lot because it's an extremely intuitive method. However, DTW is not precisely what I would call a "scalable" algorithm (but I may be wrong, in which case please correct me). Instead, I've been told I should switch to SAX.
anomaly detection for multivariate time series. For example, I acquire multiple sensor measurements from a machine in "normal operation". Then I acquire further data for the same machine. I need to identify when the behavior of the machine differs significantly from what I considered "normal operation".
Having a look at Mueen and Keogh's tutorial on the Matrix Profile, I thought it could be of help (or at least time series motifs may be of help) for my goals. However, I'm not familiar with this kind of algorithms for time series analysis (I'm not an expert of time series in general). Thus, before embarking on a time-consuming literature study, I'd like to know:
- am I right? Are these tools useful for my goals, or are there better approaches?
- can you give me the intuition about these tools? Of course I don't expect you to show me a detailed implementation, but something which could help me digest faster the relevant literature.
