# Anomaly detection with complex seasonality

I am working on a anomaly detection for a batches of daily time series (non-hierarchical) that exhibit both yearly and weekly seasonality. I tested a few algorithms and it appears that tbats() from the forecast package works best. I find it captures weekly seasonality and the trend captures the yearly seasonality via levels.

I have to split my data into the most recent 90 days (this data is not "finished" and is still having revenue trickle in). As such, I go back 365 days to get a historical trends. This denotes the "historic" and the "recent data sets" data sets. I will define recent data points that are way below the tbats value as outliers.

#Load Libraries
library(data.table) #Data cleaning Library
library(lubridate)  #Date cleaning Library
library(dplyr)      #data manipulation Library
library(zoo)        #time series indexing Library
library(stats)      #Statistics Library
library(timeSeries) #Time Series Library
library(forecast)   #Forecasting Library
library(purrr)      #data cleaning library

#orders dates oldest to newest for each department
orderedhistoric <- arrange(historicdata,Department,ServiceDate)
orderedrecent  <- arrange(recentdata,Department,ServiceDate)


This just arranges it so that the data is ordered for the ts() function

#splits into a list of department data
historicsplit <- split(orderedhistoric, orderedhistoric[,2], drop = FALSE)
recentsplit  <- split(orderedrecent, orderedrecent[,2], drop = FALSE)


This splits it by product (my 2nd column)

#Creates timeseries elements for every department
historictimeseries <- lapply(historicsplit,function(x) ts(x[3],frequency=7))
recenttimeseries <- lapply(recentsplit,function(x) ts(x[3],frequency=7))


This makes it the daily timeseries data with weekly seasonality

#finds optimal TBATS from historic data
TBATS <- lapply(historictimeseries, function(x) tbats(x))


I am running into trouble on fitting these tbats parameters to the corresponding recent data. I basically want to do a standard cross validation on this, despite it being time series data. The recent data hasn't had all the revenue posted (in fact this is what it's trying to find), so I definitely don't want it to influence the tbats parameters.

Alternatively, I would take other suggestions on how to do this. Again, I am using TBATs as opposed to auto.Arima due to complex seasonality.

• Shoot, this should be at regular stack exchange, since it's more of a coding question. Apologies – Ben Feb 14 at 19:13