As a beginner to time series analysis, I'm trying to understand the best way of detecting the points at which my univariate time series shows a change in trend direction (see highlighted example).

enter image description here

I believe these are known as 'changepoints' and/or 'step changes' (I'm not totally sure if these two terms mean the same thing, and if not, which one of these I'm trying to find)?

I've had a go at doing some simple window-based thresholding using first-order difference of the time series, where I check to see if the range within the window is greater than a certain percentage of the range across the entire dataset, but whilst this works well on this particular dataset, it is susceptible to noise (tested against other datasets).

I've noted that the spikes in the rolling standard deviation correlates with the observed change points (this makes sense to me), but I'm not sure if/how I could utilise this to produce a more robust detection solution?

Solutions in Python would be preferred, but even just theory suggestions would be appreciated!

  • $\begingroup$ Do you consider those points to be a change in trend? My visual examination of your data is that there is no trend, but there are occasional events or outliers (do you know of an explanation?). $\endgroup$ – Chris Umphlett Aug 14 '18 at 13:26
  • $\begingroup$ See Chow tests. $\endgroup$ – Skander H. Aug 14 '18 at 17:47
  • $\begingroup$ @ChrisUmphlett apologies on reflection the use of phrase "change in trend" that I explained these points denote is not correct as you've highlighted. The overall trend does in fact remain the same throughout the time-series (which is what I eventually want to go on to model) - my issue was how best to identify and remove the outliers highlighted, so that I am able to model the trend more closely without these outliers impacting on this. Thanks. $\endgroup$ – Connor Goddard Aug 21 '18 at 6:46

Some test data that has some similar properties, code is in R:


enter image description here

To analyze this: library(EnvCpt) out=envcpt(y,models="trendcpt") cpts(out$trendcpt) # gives changes at 250, 500, 750 as simulated.

plot(out$trendcpt)enter image description here

The envcpt function can fit several models and compare the fits with and without changepoints so this is why we specify models="trendcpt" so it only fits the single model.

This can be run from Python using rpy2 or alternative packages that can call R from Python. Unfortunately we don't have a Python implementation yet.

  • $\begingroup$ The downside is that although the slope is constant across the segments, the code fits a new slope after each changepoint. $\endgroup$ – adunaic Aug 17 '18 at 14:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.