Take a look at this photo:


It depicts a box plot of series of identical runs for successive i values. (AFAIK it's the standard Min/Max and 1rst, 2nd, 3rd quartiles.) So the x-axis of 1 represents 1000 runs where i=1; and the second plot shows 1000 runs where i=2; and so on.

It's easy to eye-ball and see that there's a split between the i=1,2 and i=3-19. The values for i=2 are on 'average' larger, by a little bit.

What I aim to do is given the input that produced this graph, programmatically find that split (between 2 and 3) where there's the sudden consistent change. (Step 1) It was also be awesome if there was some sort of confidence score to go along with it - just for user feedback. The change may be up or down, but I know that on both sides of the split the values will be consistent (just like for i > 2 the box plots stay pretty even and don't return to i<2 values).

Then, after that, I want to take a measurement for an unknown i and decide which 'side' of the split it falls on. Now I know I could never know that answer conclusively from a single measurement, so I plan to take several (5? 50? 100?) measurements for this unknown-but-unchanging i value. Then using those measurements know which side of the split the i falls on (Step 2). Again, it would be awesome if there was a confidence value associated with this decision.

I'm working in python so if there's a library awesome, but I'm cool with implementing an algorithm/equation myself. What's the techniques/equations/papers I should read up on to learn how to do this?

  • $\begingroup$ Your title says "change of mean", but the boxplots don't show the means. Maybe it won't make much difference, but do you mean a change in the median? $\endgroup$ – mark999 May 22 '11 at 11:42
  • $\begingroup$ My original title (edited by another person) was "What is the Statistical Method I should use to locate the change in the 'average' of successive series of data?" 'average' was quoted because I didn't know the correct term to use. $\endgroup$ – Tom Ritter May 22 '11 at 12:45
  • $\begingroup$ My bad, sorry for that. $\endgroup$ – user88 May 22 '11 at 16:33
  • $\begingroup$ why is this tagged as "time-series"? It doesn't seem correct. $\endgroup$ – Tom Reilly May 24 '11 at 17:48

If I understand you you correctly, you might need to learn about multiple comparisons:


The choice of a particular procedure is a different question, e.g., Scheffe vs. Tukey vs. Bonferroni.

At least in this framework, there is a clear and straightforward way to have hypothesis testing as well as confidence interval estimation.

  • $\begingroup$ So yes, I know that using the outcome of Step 1 (finding the split) may be incorrect and thus would throw off the result of Step 2. I'm cool with that because I can test the result of Step 2 experimentally and see if my assumptions were correct or not. But I still don't know the 'correct' way of doing either of these steps. I was hoping there was a real statistical method for doing this; as opposed to just making one up myself that works but isn't as robust as it could be. $\endgroup$ – Tom Ritter May 23 '11 at 14:52

The answer to your question can be found deep in http://www.unc.edu/~jbhill/tsay.pdf and easily available from software like AUTOBOX (which I have helped develop) and elsewhere. What you have is a sequence of medians from 1 to 19 and what you want to do is to somehow discriminate between the first k of these medians and the remaining 19-k medians. Searching for break-points is an iterative process sometimes requiring pre-filtering to deal with ARIMA structure. One has to pre-specify the minimum number in a class in order to determine the number of classes. If you specified "3" then no conclusions could be drawn about when the regime shift took place. In the other hand if you specified a "1" then one might conclude that a number of break-points were found (2,3,4,9,10,11,12,19) . Given that you specify a "2" , it is fairly obvious to both your eye and to AUTOBOX and to R. Tsay's procedure that a significant change occured at period 3. If you would like to post the 19 medians, I will post an analysis of the data.

  • $\begingroup$ I think you've overlooked the comment of your colleague @Tom Reilly: why is this tagged as "time-series"? It doesn't seem correct. $\endgroup$ – Nick Cox Jul 29 '13 at 14:44
  • $\begingroup$ Nick, A ts is a collection of transactions.As such this is not a time series but rather a sequence of observations.The fact that it is not a time series implies that any ACF/ARIMA structure would be incorrect. Intervention Detection looks for a break point in the mean/median/whatever/ to enable "single-dimension cluster analysis".What we have here are 19 values not taken at fixed intervals of time/distance but sequentially.The question on the floor is how to search for and find that point in the sequence which provides the greatest any separation.See autobox.com/cms/index.php/blog $\endgroup$ – IrishStat Jul 29 '13 at 17:13
  • $\begingroup$ I have just two points. 1. Your own answer and comment don't seem consistent (e.g. mention of ARIMA in the answer, denial that this is a time series in the comment), so your advice appears contradictory; perhaps this is just misunderstanding. 2. The original poster left unclear precisely how the data were generated, which remains problematic as the different recipes for analysis suggested do indicate. $\endgroup$ – Nick Cox Jul 29 '13 at 17:18
  • $\begingroup$ Nick , I said "Sometimes" , I should have added that pre-filtering is not applicable in this case. Hope this helps. Perhaps you and I should dialogue aboiut this offline. Please send an email to ny contact info. $\endgroup$ – IrishStat Jul 29 '13 at 17:22
  • $\begingroup$ Thanks for your clarification; I don't have anything to add to what I've said. $\endgroup$ – Nick Cox Jul 29 '13 at 17:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.