2
$\begingroup$

Probably, someone who's into technical analysis of share prices eats stuff like this for breakfast. Me, I couldn't devise a theoretically acceptable approach.

I have this thing (private working set, to be exact) that gains size across time – bad.

I have a little program that collects a time series and I am able to graph it in excel and identify trends. However, I would like to automate this (as I am keeping track of so many processes, I can’t visually see all graphs sometimes. Plus, I would like to know in real time if a problem is there, not when my test is done and I have the time series).

The statistics is what baffles me. Let me present a graph: example results to be analyzed

Each line has a letter:

A. Is OK, size does not increase.

B. Is also OK. Size increases as a product of work but it goes back to baseline when work is done

C. Obviously bad, size increases without giving back

D. Tricky one, normally a flat line, except for some small increases. The overall trend is to increase – bad

E. Even more tricky, it gives back a little but not enough to compensate the increase

F. Gives and takes sporadically, with an overall trend to increase

Was wondering if you could advise on an algorithm to analyze the timeseries that will find those “gains” as they happen.

Edit - 24/10/2014/9:51AM (more info): I talked to a guy who's properly educated in sciences and he suggested that I treat every type of graph as a "profile". Each profile will have its own detector because it's hard for one formula to fit all types of increase trends.

The profiles I am going to add so far:

A. A cap. Let's say no process should go over 100MB (that's my case, your's will vary). This is a simple way to ensure that no matter the profile, the big increases will be caught. Some smaller increases may fly under the radar.

B. A profiler for C-graph - where there is simply not give-back, just increase. I actually have quite a few of those so this simple check is quite valid.

C. For D and E, I might try something like establishing a baseline and over a longer period of time, make sure it is not breached. Also will have to allow for some tolerance from the baseline (we can't expect the process to go EXACTLY back to the baseline)

D. For F, I think I want to average out every so-and-so readings to create a line like C and see the slope's angle.

Any ideas will be welcome

Thanks

$\endgroup$
  • $\begingroup$ Welcome! I added the quality-control tag to give you some pointers to a collection of highly relevant techniques. $\endgroup$ – whuber Oct 23 '14 at 17:48
  • $\begingroup$ Can you clarify the following: Do you consider F bad? Is the data you showed, all you usually have or do the time series switch behaviour, e.g., could a time series start like B and then switch to E at some point and this is what you want to detect? $\endgroup$ – Wrzlprmft Oct 23 '14 at 20:36
  • $\begingroup$ @Wrzlprmft, Sure, thanks for picking up. 1. F is bad. It gives and takes but the overall trend is to increase. 2. Let's assume the time series doesn't switch behavior. Most of it is either B or D, but not on the same process. Usually, each process only follows one pattern. 3. I want to detect all those that, as a trend, increase steadily. I have more info that I will edit in. $\endgroup$ – DraxDomax Oct 24 '14 at 8:49
  • $\begingroup$ @whuber, do you mean that I have some extra information lying around? Because I can't find it. Thanks for the welcome and attention. $\endgroup$ – DraxDomax Oct 24 '14 at 8:49
  • $\begingroup$ There is a huge literature on this problem: research statistical quality control and statistical process control. Among many possible solutions, a combined Shewhart-CUSUM control chart will very quickly detect every one of the trends illustrated in your post. (I wrote software to do this 25 years ago and during the next decade reviewed over a million such charts that it created.) $\endgroup$ – whuber Oct 24 '14 at 14:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.