1
$\begingroup$

I have some time series that is a list of positive numbers $x_1, x_2 ... x_n$. These numbers are ordered in time, and equally spaced, and the order between the number is important.

Now, I would like to identify some consecutive numbers in that time series that are unusually high. For example, if I have this series: 2,1,1,2,1,1,2,1,3,5,6,7,3,2,1, I would like to find that 5,6,7 are some consecutive numbers that are unusually high in that time series. These consecutive numbers represent something unusual.

Thus, my questions is what kind of mathematical models can be used to identify some consecutive numbers that are usually high in a time series? Some simple idea could be to just look at the average and look for numbers that are higher than the average + standard deviation. But it sounds a bit simple. Is there some other, perhaps better approaches to this problem?

About my data, I do not make any assumptions except that this is a time series of positive numbers. I also do not know the distribution of the data.

Note: I have previously asked a similar question on StackExchange. But this question is different. I want to identify an interval of consecutive values (in that question) rather than a single value (in my previous question).

$\endgroup$
2
$\begingroup$

Online mean shift algorithms contains a discussion and some of my comments on this very important issue.

Essentially one wants to specify (1) the minimum # in the new group and (2) the magnitude of the shift required while (3) specifying a level of confidence. For example I have N observations and I wish to identify the beginning and end time interval (1) where the mean shifted by at least "z" percent (2) of the overall mean with a confidence of x percent (3).

This is a common problem in with our supply chain customers where fraudulent activity re-directs product to another site/accounting application with a resultant local mean shift. This is often referred to as "diversion"

Non-stationarity is a symptom with possibly many causes. One cause is a shift in the mean at one or more points in time or a change in trend . Another possible cause is a change in parameters at one or more points in time. Another cause is a deterministic change in error variance at one or more points in time.

As an example consider the following time series (55 monthly values)

enter image description here . In order to test for a level shift , one needs to form a model and verify the assumptions underlying the selected model. In this case we have a simple level shift at period 37 . enter image description here with equation here enter image description here and in more detail here enter image description here

The residual plot is here enter image description here .

Thus we can conclude that there are two statistically significant means ... The first for period 1-36 ... The second for period 37-55 .

Upon closer inspection we find a possible second explanation. Closely examine the residual plot and notice that there appears to be a difference between the mean of the residuals for period 1-19 vs 20-36 suggesting a secondary mean shift. Closely examining the original plot in this area we get enter image description here

The reason that AUTOBOX did not detect/report two level shifts suggesting 1-17 vs 18-36 vs 37-65 was that this potential second level shift was not statistically significant at the default level of confidence.

In summary ...

  1. Pulses in the data need to be identified and rectified
  2. Model errors need to be Gaussian i.e. free of evidented structure in order for parametric tests to be conducted. In this case an ar(1) was needed with 4 pulse indicators.

    Finally we can "see" that there may ALSO be a need to adjust for changing model error variability as the dispersion around observations 37-55 is visibally more pronounced than periods 1-36 suggesting a weighted least squares approach as suggested by

http://docplayer.net/12080848-Outliers-level-shifts-and-variance-changes-in-time-series.html

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.