The time series data, as shown in image 1, has 3 separate steps connected by increasing data. How do I detect these steps?enter image description here

An easier set of data with one step could be created in R as follows:

x <- seq(1,220)
y <- c(seq(1,100),rep(100, times = 20),seq(101,200))
plot(x,y)

The output is shown in image 2. How would I detect this step? I think a solution to this simple problem would be similar to finding the steps in the earlier dataset.enter image description here

up vote 0 down vote accepted

There are many ways. For the sample with R code, it's as simple as differencing then analyzing the new series, e.g. see this code and the plot:

dy=diff(y)
plot(x[2:length(x)],dy)

enter image description here

This may not work for the real data though, if the data generating process is different from your sample code.

  • It does turn into a step problem if I smooth it first using moving averages. Would you happen to know how I find the optimum window size for moving averages? For example, how do I defend using a window size of 5% of the data vs 10% of the data? – SPV May 17 '17 at 2:53
  • I don't think that the window size can be wider than the step width. – Aksakal May 17 '17 at 3:20
  • @SPV: A rolling mean is essentially a low pass filter. That is, it attenuates the amplitude of frequencies above a certain point. A filter like this can therefore be used to smooth out high frequency noise, while retaining a lower frequency signal. In general terms, you want the window to be of a length so that it attenuates as much noise as possible, without distorting the signal you are interested in. As such Aksakal's tip is a reasonable one. – AkselA May 17 '17 at 10:24

Your Answer

 
discard

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.