4
$\begingroup$

I am helping a friend with a data project. He's interested in building a canary-in-the-coal-mine alert system for his website which tells him when the number of users dips below some critical lower bound. The number of users varies by time of day, and day of the week. I have 550 days of by-the-minute data, though we can probably aggregate that into 5 minute buckets to make it less noisy at the cost of early warnings. Suggestion on how to pick the right size bucket would be helpful.

This is what the smoothed daily fluctuations look like by DOW:enter image description here

There appear to be two cycles each day, though there's typically not much of dip in between them. There's also a positive trend over time that I need to take into account:enter image description here Obviously, the 4am-9pm period in the first graph should look the same over time, but the rest of the curves have been shifting up and becoming steeper.

I am having trouble getting the shape of the daily curve right. I think this calls for some sort spline with 4-6 knots, but I am not sure how to integrate this with the time-series aspect. Approaching it from the other direction, I've been trying to fit this into an unobserved components model (UCM) in Stata, but I am lost as to how to specify the trend, seasonality, and cyclicality, and how to treat the exogenous variables. The few things I'be tried either look off or have not converged in a reasonable amount of time. The other issue is how to translate the RMSE of the UCM forecast into an interval of some sort, or at least the left end of one. This is much more complicated than any time-series model I've built before, so any modeling advice would be appreciated.

Improve this question
$\endgroup$
1
  • $\begingroup$ Have you considered an ARIMA model? Stats might have some nice features to tune the parameters interactively until you have some acceptable model. Given your example you will have to play with all the 3 components, AR, MA and detrending. $\endgroup$
    – Robert Kubrick
    Sep 11, 2012 at 16:44

4 Answers 4

Reset to default
2
$\begingroup$

Basically, you're looking for outliers in a time series. You're most interested in low outliers (but high outliers might be interesting too). This site already has a lot of information posted on this topic, but I'd like to specifically point you to this answer by Rob Hyndman:

The basic idea is to find robust estimates of the trend and seasonal components and subtract them. Then find outliers in the residuals.

(link to question)

In your case, it's a little trickier, as there's a 2 "seasonalities" in your data: minute-of-the-day and day-of-the week. One place to start might be a simple linear regression with dummy variables for minute-of-the-day and day-of-the-week and a simple linear trend variable (number of minutes since the start of the dataset). Fit this model to the data, and then look for outliers on the residuals.

Once you've got that working, you can look for more robust estimates of the 2 seasonal periods and the trend, using either loess smoothers, splines, or some variation of the stl function.

Improve this answer
$\endgroup$
0
$\begingroup$

I only add remark about modeling the shape of daily average number of users per minute - you can decompose that curve using sum of two gaussian functions, such sum will have 6 free parameters to fit.

http://en.wikipedia.org/wiki/Gaussian_function

or father sum of one rescaled gamma/beta distribution function and gaussian function. Gamma or beta seems to be batter choice for that first very flat peak around 14:00.

And ahter fitting sum of two functions you can observ how they parameters change depending on the day of weak.

For fitting you can try BGFS algorithm, but the problem will be finding staring values of parametres, but in tis case it can be resolved very easy - just cut daily profiles of number of users into two sets - first for 10:00-15:00 and second 15:00-2:00m then fit one gaussian or rather gama/beta to first set and one gaussian function to second one, then use fitted parametres as starting values for BFGS for fitting sum of two choosen distributions.

But I don't see the point of modeling daily average numbers of users. The written-on-the-knee model for forecasting number of users is to take weighted average of number of users and daily profiles of average number of users... I can write more in this topis if you want.

Improve this answer
$\endgroup$
0
$\begingroup$

I'm not sure you need to do something incredibly sophisticated to solve this problem. One simple solution might be to just group the five minute time periods over the past month or so, calculate the mean and variance of the number of users in each time block. When the number of standard deviations is one or two from the mean, issue the warning. You could make adjustments so that when the number of users is consistently low, like at four AM, then the warning is not issued.

Improve this answer
$\endgroup$
2
  • $\begingroup$ This seems like reasonable advice, although I would be concerned about two particular aspects. 1) Rules like X standard deviations shouldn't be relied upon, the metric should be evaluated upon out of control sequence length, false positives and more general resource constraints. 2) I'm not sure if this is sufficient to account for the upward linear trend in the data, although since the objective is to see "too low counts" perhaps that is not a big issue. $\endgroup$
    – Andy W
    Sep 12, 2012 at 12:57
  • $\begingroup$ Well if you're worried about robustness, you could do something like bottom or top x% instead. If you're really worried about the trend, you could take the ln daily average number of users, regress it against a time trend (over a relatively short period, to avoid the problems you mentioned before about the change in the trend over time). The daily residuals should be without trend and you should be able to create a daily adjustment factor that you could apply to all the five-minute groups that could make the five-minute periods without a daily trend. $\endgroup$
    – John
    Sep 12, 2012 at 14:07
0
$\begingroup$

A few thoughts -

  1. Single-class SVMs can give better results when the purpose of your model is to identify outliers within noisy data - so long as you tune the misclassification slack parameter appropriately. It directly addresses the problem of "build a model to classify healthy and unhealthy behavior".
  2. Daily/Yearly seasons are cyclical, and thus can be modeled nicely using spectral components of time - Sin(x), Cos(x), Sin(2x) etc... This will likely give better results than the spline fit you mentioned.
  3. The ongoing non-seasonal trend, e.g. you friend's increasing popularity over time, is a tricky part. You might simply retrain the model every night to predict the next day. However, it is easy to account for being more popular over time while maintaining alerting on drops on traffic (i.e. the "server down" scenario) since no special behavior is needed!
Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.