I'm looking for an algorithm that can help identify abnormal trends in timeseries metrics. We offer several services which we monitor active usage against, for any given time of day (typically updated once a minute). Here's a example of one of our services:
We do have systems in place like Nagios to detect major issues. But what I'm more interested in is gradual degradation that doesn't get noticed until ... well, it gets noticed. For example, if we zoom in to 12/6 we see:
Here we can see the abnormalities on 12/1 and 12/4 during peak. Are there any formal methods of detecting this sort of trend (or rather, abnormaility to a historical trend)? Specifically interested in the trend here, not the raw numbers. So for example, these services have a hisotrical decreasing usage as time goes on - eventually, these numbers will drop from say 60k peaks to 15k peaks.