0
$\begingroup$

Apologies if this question has already been asked, but a lot of similar questions are regarding R or complex algorithms that I don't want.

I have numerous 2-dimensional time series, eacch plotting the average success percentage of a task. Occasionally, some certain task which previously worked at a high rate (eg. 60-90%) starts to fail often (due to some unforeseen event), so the average success rate drops significantly (eg. 0-30%).

I want a simple algorithm for a program to detect and notify me if this drop/change point occurs. I don't need to locate when, I just need an alert to see which one of these plots detected a change point (aka the task is starting to fail).

I've seen a lot on CUSUM or other methods that require preset thresholds, but I can't preset a specific threshold, since some of these plots (or task) start at a higher percentage, and starts to fail more than other plots.

What are some simple algorithms to detect a change-point/drop in a time series? Or what are some ways to detect a significant change in the average success rate (mean) plotted by time?

Improve this question
$\endgroup$
3
  • $\begingroup$ are the averages for series known in advance? $\endgroup$
    – Aksakal
    Commented May 21, 2020 at 0:50
  • $\begingroup$ every time the task is performed (success or failure), the average of overall success rate would be updated. $\endgroup$
    – brianmchoi11
    Commented May 21, 2020 at 1:00
  • $\begingroup$ if you have mean shifts, then unconditional means do not make much sense. but you answered my question $\endgroup$
    – Aksakal
    Commented May 21, 2020 at 1:03

2 Answers 2

Reset to default
0
$\begingroup$

If your data is distributed in a well-defined way, Westgard rules are probably the simplest solution. For your problem, I'd propose something like this:

  1. Compute some lower bound on "good" periods in your time series, e.g., the 10% percentile.
  2. Work through previous failure-cases and establish a sequential-deviation rule that discriminates failures from usual noise. For example: three out of the last four numbers are below the 10% percentile. You could also use something like mean and SD but that rarely represents proportions well.
  3. Apply this rule for each new data point as it enters.

This is very fast, easy to understand, and does not model when the change point occurred. However, it only works well if the system is non-noisy, i.e., it should be easy to discriminate true change from business-as-usual. If not, you'd be better off going for a probabilistic approach using a changepoint package that can detect intercept changes in time series (mcp, EnvCpt, or others)

Improve this answer
$\endgroup$
-1
$\begingroup$

The simplest method is to run a Anova on past (-k,-k/2] and [-k/2+1,0] observations, and test its significance. Basically you're testing whether the mean changed k/2 periods ago. If there's a significant mean change at k/2 periods ago, then ANOVA will detect it.

Improve this answer
$\endgroup$
2
  • $\begingroup$ would this work if the data is grouped into buckets of n data? in other words, rather than the time series showing a gradual decline in average success rate once the task starts failing, the time series plot would resemble a piece-wise linear function and show a huge dip. $\endgroup$
    – brianmchoi11
    Commented May 21, 2020 at 1:03
  • $\begingroup$ when $k/2=n$ you should detect a dip. you need to play with settings though. $\endgroup$
    – Aksakal
    Commented May 21, 2020 at 1:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.