I have some specific concerns regarding data handling and aknowledge the right Time Series forecasting path. I've followed the ARIMA modelling in R guide from the Forecasting: principles and practice, which I think it's one of the best out there. Unfortunatelly most of the guides or tutorials where made with "prepared or adjusted data" so you allways get a nice outcome. For this post I've prepared a public pastebin with the DAILY data set from 2015-01-01 to 2016-09-31 I'm using. I've read a lot of posts but still seems confunsing to me.
1- Since I have skewed data, (ranging from 160.000 to 2.363.188.500, amount of daily sales in my country's currency) do I need to scale it somehow ? My goal is to forecast the next 30 days sales amounts, so is better to leave the data as it is ?
# READ FROM URL
library(data.table)
salesData <- fread('http://pastebin.com/raw/0ebKmSw5',
header = TRUE,
data.table = FALSE)
hist(salesData[,2]) # POSITIVE SKEWED TO THE RIGHT
2- I know that working with daily data has some issues for what I'd read. In Rob J Hyndman's blog here he says that:
"Unless the time series is very long, the simplest approach is to simply set the frequency attribute to 7"
I only have 612 data points, so it is safe to set the frequency to 7 ?
amounts <- salesData[,2]
startD <- as.Date("2015-01-01")
tsSales <- ts(amounts, frequency = 7, start = startD)
From here I've used this test to ensure that no transformation is needed, am I correct ?
BoxCox.lambda(tsSales) # = 0.7440572 CLOSE TO 1 -> NO TRANSF. NEEDED
tsdisplay(tsMaxi,main = "ORIGINAL DATA")
3- In this next step, do I really need to do the STL decomposition and Seas.Adjustment on my TS ?
stlSales <- stl(tsSales, s.window = "periodic", robust = TRUE)
adjSales <- seasadj(stlSales)
plot(adjSales, main = "STL DEC. & SEASADL")
4- Sticking with the example, here I check if a Seas.Differentiation is needed, the output is that I need to take a First Diff., but the Box.test states that the series is already stationary, how can I interpret this
# p-value = 2.2e-16 < 0.05 THE SERIES IS STATIONARY
Box.test(tsSales, lag = 7, type = "Ljung")
# OUTPUT OF THIS TEST: TRUE, THERE IS SEASONALITY SO IT NEEDS TO BE DIFFERENTIATED
fit <- tbats(tsSales)
summary(fit)
!is.null(fit$seasonal.periods) # TRUE
# NO. OF SEAS. DIFFERENCES REQUIRED TO MAKE THE TS STATIONARY
ns <- nsdiffs(adjSales)
if(ns > 0) {
diffSales <- diff(adjSales,lag=frequency(adjSales),differences=ns)
} else {
diffSales <- adjSales
}
tsdisplay(diffSales,main = "SEAS. DIFFERENTIATED SERIES")
5- Next, I do an auto.arima() check and this suggests that the series may have multiple seasonalities. Does this make any sense ? I'm choosing auto.arima because I'm having a hard time deciding the Arima() order by looking at the previous PACF plot.
# PACF INDICATES: AR(6)?
# DIFF. ORDER: d = 1
# MA ORDER: MA(1)?
fit <- auto.arima(diffSales) #SUGGESTS: ARIMA(3,0,1)(2,1,0)[7] (SEASONAL ARIMA)
summary(fit)
6- And finally, after a Residual Analysis, the final forecast and plot
fitResiduals <- residuals(fit)
Acf(fitResiduals) # SHOWS 3 CORRELATIONs OUTSIDE THE LIMITS
Box.test(fitResiduals, lag = 7, type = "Ljung") # p-value = 0.07246 > 0.05 ?
forecastedModel <- forecast(fit, level = 0, h = 30)
plot(forecastedModel, main = "FORECASTS")
And other finally questions are:
- How can I get to plot these forecasted values with the original series ?
- How can I interpret these weird date X-axis that is plotting ?
- I'm getting negative results in the forecasting step, why's the reason ?
Hope you can help me somehow or guide me if I'm mistaken or something. I've tried to make the best reproducible example, sadly I cann't yet upload more urls to my plots because of my low reputation. Thanks in advance.
EDIT ( DATA SET UPDATED )
I was missing some date meassures in my original data set, so here is the updated series (sorry I can't post more links, I don't want to loose the former ones in the original question):
http://pastebin.com/raw/QtqijXaK