Testing a proportion in an online setting

Question

I work in an online security setting. My goal is to detect if the number of locked accounts per time unit is stable or not. I've tried several approaches, detailed below, but I am not satisfied yet. One of the challenges is to deal with fluctuating amounts of traffic.

If total traffic would be constant over time, then a Poisson distribution could help out. The locked accounts can be seen as arrivals. Since traffic fluctuates heavily over a day, it is difficult to test per minute or hour. Testing per day could be an option. Holiday periods and weekends could show too low observed numbers, but I am mostly interested in increases.

A change detection algorithm like ADWIN could also help out. If applied on day totals, the assumption would be that the per day the number locked accounts should be stable. Days with lower traffic could be a problem like weekends or holidays; or rather the day after the weekend that show an increase. A proportion seems more to the point.

To keep track of the sample size, a chi-square test could help out. One could test proportions independent of sample size as long as cells are filled with more than 5 observations. The observations of the last hour could easily be compared to the observations of the last week since the chi-square test takes the proportions into account. Some bonferroni stuff should be in place.

I've tested a neural network for baseline traffic prediction, but this method seems unstable. It basically uses a predictor, the error variance and an error outlier rule. This should be state of art. The reason why it not so stable is that the traffic varies wildly per minute, increases and decreases of 4 times seems normal. Could averaging over hours instead of minutes make this model work better?

Would anyone have further suggestions?

Answer 1

Having a look at the data, only the freq variable (second column). Since it is a count, analyzing its square root. The autocorrelation function decays almost linearly to about zero at about lag 400, but that must be an artefact of all the nightly zeros. So, analysing only the block of data up to the first block of zeros, freq[1:934], the autocorrelation function is more sensible, nonzero only at the first lag, with value there about 0.3. Plotting that part of the series, it is strange, almost constant with values about 20, with a few values much larger than that, coming in bursts. So describing the series with an autocorrelation function is not a very good description, maybe some kind of hidden Markov process would give a better model, with one almost constant regime, and another bursting regime, and then the zero-regime. You could try that!

EDIT

Answering question in comments: count data are often poisson-distributed, at least, that is the simplest model for counts. And, the square root is the variance-stabilizing transformation for the Poisson distribution family. That is, while we have that, for $X\sim \text{Po}(\lambda)$, that $\text{Var}(X)=\lambda$, we have that $\text{Var}(\sqrt{X})\approx \frac14$, approximately independent of $\lambda$. And, many statistical methods, like least squares and most time-series models like ARIMA, works best for data which have a variance independent of its mean.