3
$\begingroup$

I am asking this with the hope this question can be helpful for me, and for others in my same situation.

I am working for a Company. We mounted some sensors on an industial machine, in order to get some data about its state in working moments (between 3 and 10 minutes), like oil pressure and temperature (taken at frequency x), or other vibrational data (this machine has some mechanical arms; these data are taken at frequency y, with y >>> x).

Now, I am supposed to retrieve these data (in form of Univariate Time Series), and empirically apply some Anomaly Detection Algorithm. Our final aim will be to apply some online mechanism on online data stream for instant detection. For now, just for the first period, I can apply some batch technique.

My question is: given all I said, which technique/algorithm is more suitable to the problem? I've seen almost all of it (Twitter's, Netflix's RAD, Farrington, all Time Series Analysis packages for R, and so on), but I cannot figure out which should I apply to get the best results. Most of them find as an outlier high peaks, for example in vibrational data, that are instead normal operations, relative to some product change or operation resets.

A nice approach I found, is the one based on SAX and bitmap image, and assumption-free, by Keogh and others (http://alumni.cs.ucr.edu/~ratana/KumarN.pdf). I found it easy and intuitive, but I have a feeling it's not what I am looking for.

I am not an expert about what this machine does, so I cannot figure out, by reading the data, what operation is it doing, or label an operation as normal or anomalous. So, an unsupervised approach is preferable.

I am using R and RStudio for the data analysis.

$\endgroup$
3
$\begingroup$

Hmm. That Keogh guy is smart and handsome, but I am not sure SAX bitmaps are right for this.

Time series discords are empirically the best (in independent cook offs). Chandola et al. (2009) found that with 19 different publicly available data sets, comparing 9 different techniques, time series discords is the best overall technique.

They are simple, fast to compute and general.

I suggest you glance at this tutorial, esp slides 25 to 30.

Chandola, Varun, Deepthi Cheboli, and Vipin Kumar. "Detecting anomalies in a time series database." Computer Science Department, University of Minnesota, Tech. Rep (2009).

$\endgroup$
1
  • $\begingroup$ Could you please mention where in the paper Chandola et al. claim that discord techniques perform better? I could not find any passage repeating this statement. In opposite, they say: "At a high level, our results indicate that none of the tech-niques are superior to others across all data sets, but havecertain characteristics that make them effective for certaintypes of data sets, and ineffective for certain other". $\endgroup$
    – Code Pope
    Apr 6 '20 at 17:19
0
$\begingroup$

First of all Identify- "WHAT IS ANOMLAY IN YOUR APPLICATION", there is no algorithm that will give u direct abnormality. they are focused on outlier detection.

1) If you can generate some data at abnormality, build a classification model.

2) Can you build a relation among variables present in data. Find a relation by NN or regression, any deviation from known relation is abnormality.

3) If outliers are abnormalities in your application. I can share some links like- https://machinelearningstories.blogspot.com/2018/07/anomaly-detection-anomaly-detection-by.html

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.