I am working on a anomaly detection for a batches of daily time series (non-hierarchical) that exhibit both yearly and weekly seasonality. I tested a few algorithms and it appears that tbats() from the forecast package works best. I find it captures weekly seasonality and the trend captures the yearly seasonality via levels.
I have to split my data into the most recent 90 days (this data is not "finished" and is still having revenue trickle in). As such, I go back 365 days to get a historical trends. This denotes the "historic" and the "recent data sets" data sets. I will define recent data points that are way below the tbats value as outliers.
#Load Libraries library(data.table) #Data cleaning Library library(lubridate) #Date cleaning Library library(dplyr) #data manipulation Library library(zoo) #time series indexing Library library(stats) #Statistics Library library(timeSeries) #Time Series Library library(forecast) #Forecasting Library library(purrr) #data cleaning library #orders dates oldest to newest for each department orderedhistoric <- arrange(historicdata,Department,ServiceDate) orderedrecent <- arrange(recentdata,Department,ServiceDate)
This just arranges it so that the data is ordered for the ts() function
#splits into a list of department data historicsplit <- split(orderedhistoric, orderedhistoric[,2], drop = FALSE) recentsplit <- split(orderedrecent, orderedrecent[,2], drop = FALSE)
This splits it by product (my 2nd column)
#Creates timeseries elements for every department historictimeseries <- lapply(historicsplit,function(x) ts(x,frequency=7)) recenttimeseries <- lapply(recentsplit,function(x) ts(x,frequency=7))
This makes it the daily timeseries data with weekly seasonality
#finds optimal TBATS from historic data TBATS <- lapply(historictimeseries, function(x) tbats(x))
I am running into trouble on fitting these tbats parameters to the corresponding recent data. I basically want to do a standard cross validation on this, despite it being time series data. The recent data hasn't had all the revenue posted (in fact this is what it's trying to find), so I definitely don't want it to influence the tbats parameters.
Alternatively, I would take other suggestions on how to do this. Again, I am using TBATs as opposed to auto.Arima due to complex seasonality.