I work in an online security setting. My goal is to detect if the number of locked accounts per time unit is stable or not. I've tried several approaches, detailed below, but I am not satisfied yet. One of the challenges is to deal with fluctuating amounts of traffic.
If total traffic would be constant over time, then a Poisson distribution could help out. The locked accounts can be seen as arrivals. Since traffic fluctuates heavily over a day, it is difficult to test per minute or hour. Testing per day could be an option. Holiday periods and weekends could show too low observed numbers, but I am mostly interested in increases.
A change detection algorithm like ADWIN could also help out. If applied on day totals, the assumption would be that the per day the number locked accounts should be stable. Days with lower traffic could be a problem like weekends or holidays; or rather the day after the weekend that show an increase. A proportion seems more to the point.
To keep track of the sample size, a chi-square test could help out. One could test proportions independent of sample size as long as cells are filled with more than 5 observations. The observations of the last hour could easily be compared to the observations of the last week since the chi-square test takes the proportions into account. Some bonferroni stuff should be in place.
I've tested a neural network for baseline traffic prediction, but this method seems unstable. It basically uses a predictor, the error variance and an error outlier rule. This should be state of art. The reason why it not so stable is that the traffic varies wildly per minute, increases and decreases of 4 times seems normal. Could averaging over hours instead of minutes make this model work better?
Would anyone have further suggestions?