0
$\begingroup$

I have data values for events per second (EPS) present in log files pertaining to various devices. The idea is that these values should help us observe a trend and create thresholds for specific times during the day, and specific weekdays. If the values observed in the past hour exceeds the threshold, an alert is generated.

What we have tried: We have tried using a trimmed mean method to average out the EPS values for specific subsets of time and day. However, the data is more chaotic than we thought and the thresholds are falling short such that we have a lot of false positives. So we are now looking at machine learning algorithms to see if they offer better performance.

The idea is to fit a model to the past data and attempt to predict future values. Alerts are generated on noticing deviations from the predicted values coming out from the machine learning algorithm. We have looked at 'SVM', 'MLP', 'Neural Networks' etc., however we do not know which approach would work best since we do not have significant data science knowledge. Any recommendations are appreciated.

Our data looks like this:

Hour,Weekday,Month,Value 14,Wednesday,June,1456.345 ....

Is there a tool (like Weka or libsvm) that would allow us to input this data, train the model, and make future predictions?

$\endgroup$
1
$\begingroup$

From what you explain, you have : intra-daily and intra-weekly. We have had previous questions on this. Searching here for "multiple seasonalities" will yield a couple of posts that may be enlightening.

I'd recommend using the tbats() function in the forecast package for R, which can deal with multiple seasonalities. I'd use the data up to but not including the data point you are interested in to train a model, then calculate a high quantile forecast, say 95%, then check whether the actual observation falls above that quantile.

Let's run an example. We first simulate four weeks of hourly data with both intra-daily and intra-weekly seasonality and a slight upward trend and store this as a msts object (which stands for "multiple seasonal time series" - see ?msts):

library(forecast)

time <- 1:(4*168)
set.seed(1)
xx <- msts(sin(2*pi*time/24)+c(1,1,1.2,0.8,1,0,0)[((time-1)%/%24)%%7+1]+
    time/400+rnorm(length(time),0,0.2),seasonal.periods=c(24,168))

plot(xx)

multiple seasonal time series

Next, we fit a TBATS model. This step can take a while if you have more data, or it can even hang your R - if so, I'd recommend taking only the last few weeks of data.

model <- tbats(xx)

You can inspect the components of your TBATS model using tbats.components():

tbats.components(model)

Time Series:
Start = c(1, 1) 
End = c(4, 168) 
Frequency = 168 
             observed     level         slope       season1      season2
1.000000  1.134778283 0.7732623 -6.515840e-04  0.2648350787 -0.138887537
1.005952  1.539228665 0.7643020 -1.301022e-04  0.4956773250 -0.059885259
1.011905  1.543731059 0.7622478 -9.689533e-06  0.6878854679  0.015076437
1.017857  2.190081564 0.7495647  7.692112e-04  0.8374512716  0.086414814
1.023810  2.038077381 0.7450848  1.078842e-03  0.9304295810  0.149071774

Now we calculate a mean and a quantile forecast using forecast.tbats(). Note that if we set level=90, we get a 90% prediction interval bounded by a 5% and a 95% quantile forecast, which is what we want - since we are only interested in larger than expected outcomes, we will simply disregard the lower bound.

fcst <- forecast.tbats(model,h=24,level=90)
fcst

         Point Forecast     Lo 90    Hi 90
5.000000      1.5490222 1.1676584 1.930386
5.005952      1.8802119 1.4963631 2.264061
5.011905      2.1672261 1.7832911 2.551161
5.017857      2.3899943 2.0057284 2.774260
5.023810      2.5502875 2.1656541 2.934921

Finally, we can plot the forecast, including the prediction intervals:

plot(fcst)

As we see, the thresholds will be quite different during different times of day (and different days of the week).

TBATS forecast

$\endgroup$
  • $\begingroup$ Thank you for your detailed and thoughtful response. We would very much like to try the tbats() function in R (although, as you mentioned, it is known to be slow). However, we are new to R and wondering whether we need to transform or format the data in the CSV file before we load it in R. Currently, the data in the CSV file is in the format shown in the question above. We did try looking for data input format for this function online but did not end up with anything useful. $\endgroup$ – learnerX Jun 8 '16 at 15:40
  • $\begingroup$ OK, we were able to use the msts function to format the serial values data in ascending order of time into seasons. We set seasonal.periods=c(24,168) as in your example. The predictions were really close to the actual values observed. However, we were wondering what is the significance of the values '24' and '168'. $\endgroup$ – learnerX Jun 8 '16 at 16:44
  • $\begingroup$ I am guessing the '24' refers to number of hours in day (24 seasons) and the 168 refers to 7 weeks of data? $\endgroup$ – learnerX Jun 8 '16 at 16:56
  • $\begingroup$ Yes. You can look at the documentation with ?msts, and you will see that seasonal.periods is supposed to be fed "A vector of the seasonal periods of the msts." That is, one season is 24 periods long, and the other 168 periods - corresponding to daily and weekly cycles for hourly data. $\endgroup$ – S. Kolassa - Reinstate Monica Jun 8 '16 at 19:17
  • $\begingroup$ Re csv files: I'd recommend including a date field to be able to differentiate the different Wednesdays in June, since there is more than one. You will need to read it into R as a character (look at the colClasses argument to ?read.table), then convert it with as.Date. Or combine the hour and date into a timestamp and look at ?POSIXct. $\endgroup$ – S. Kolassa - Reinstate Monica Jun 8 '16 at 19:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.