1
$\begingroup$

What algorithm should I go for if I want to determine collective outliers within a dataset?

By collective outliers, I mean a series of data points differ significantly from the trends in the rest of the dataset. The data refers to time-series. I cannot find an algorithm that does this. The ones that I find are all focused on finding point outliers.

$\endgroup$
6
  • $\begingroup$ What do you mean by “collective outliers”? What is your data? $\endgroup$
    – Tim
    Aug 15, 2022 at 13:18
  • $\begingroup$ @Tim Collective outliers = A series of data points differ significantly from the trends in the rest of the dataset. However, the individual data points within a collective outlier may not seem like a point outlier or a contextual outlier. $\endgroup$
    – Iamtrying
    Aug 15, 2022 at 13:20
  • $\begingroup$ I am not talking about a data set in particular, I just want to know what algorithm I need to use when I want to detect this type of outliers. $\endgroup$
    – Iamtrying
    Aug 15, 2022 at 13:22
  • $\begingroup$ But you mean time-series data? Or spatial data? If not, what do you mean by "context"? $\endgroup$
    – Tim
    Aug 15, 2022 at 13:24
  • $\begingroup$ Time-series data. $\endgroup$
    – Iamtrying
    Aug 15, 2022 at 13:26

1 Answer 1

0
$\begingroup$

I would suggest first trying standard time series outlier detection, e.g. tsoutliers or anything based on the difference between the time series and its smoothed version. Those methods usually also detect groups of outliers, as long as those are not too large.

As a next step, you could also play around with the smoothing parameters or those methods. Or just use a rolling average of your time series with a window size somewhere near the maximum size of an outlier group you are willing to accept and then, again, feed those to the above outlier detection methods. The rolling average will have the effect of compressing the time series into a series of groups of points. The variation of the smoothed series is then, of course, reduced, but groups of outliers will create a larger deviation.

If this is still not enough, you might have to try to define the type of outliers you are looking for more precisely. That could also be done by creating lots of examples that do have those outliers and lots which don't. Then you can feed it to a supervised machine, a binary outlier classifier.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.