# Fitting time series with outliers

I have daily sales data for a department store for the past 850 days. I have indicators on the major holidays and the days leading up to the major holidays. The number of days before the holidays that are included was chosen by AIC. The issue I'm having is that there are outliers throughout the data that I'm not sure how to handle. Or, at least that's what I think is happening since I don't seem to get accurate forecasts. I'm using a CV to calculate the MAPE of forecasts two weeks out, using the first 450 days as the initial training set and the rest to see how well the model forecasts the data.

I've used tso() from the tsoutliers package and tsoutliers from the forecast package to find outliers. They both give different results.

tsoutliers(data$Sales)$index
[1] 230 270 271 328 635

$replacements [1] 2222.160 2088.573 2231.577 1812.380 2138.655 train = 454 trainingdata = data$Sales[1:train]
trainingdata = ts(trainingdata,frequency = 7)
tso(trainingdata,types = c("AO", "LS", "TC"))

Series: trainingdata
ARIMA(2,1,1)(2,0,0)[7]

Coefficients:
ar1     ar2      ma1    sar1    sar2      AO52      TC68       TC80      AO86
0.2872  0.1331  -0.9717  0.3567  0.4607  885.2061  890.3690  -863.4296  836.8638
s.e.  0.0508  0.0480   0.0107  0.0436  0.0429  169.2521  163.4243   166.0282  169.8535
AO111     AO121      TC229     AO259      TC270     AO328     AO416
754.1791  691.0849  1236.8523  711.3954  1790.0292  764.9712  920.1783
s.e.  169.2042  167.7273   163.1458  167.9835   163.9663  170.0103  168.9235

sigma^2 estimated as 44080:  log likelihood=-3064.92
AIC=6152.24   AICc=6153.65   BIC=6222.21

Outliers:
type ind  time coefhat  tstat
1    AO  52  8:03   885.2  5.230
2    TC  68 10:05   890.4  5.448
3    TC  80 12:03  -863.4 -5.200
4    AO  86 13:02   836.9  4.927
5    AO 111 16:06   754.2  4.457
6    AO 121 18:02   691.1  4.120
7    TC 229 33:05  1236.9  7.581
8    AO 259 37:07   711.4  4.235
9    TC 270 39:04  1790.0 10.917
10   AO 328 47:06   765.0  4.500
11   AO 416 60:03   920.2  5.447


Running BoxCox on the data it recommends a transform of the data

lambda <- BoxCox.lambda(data\$Sales)
trainingdata = BoxCox(trainingdata,lambda)
tso(trainingdata,types = c("AO", "LS", "TC"))
Series: trainingdata
ARIMA(3,1,1)(2,0,0)[7]

Coefficients:
ar1     ar2      ar3      ma1    sar1    sar2      LS3    AO52     AO53    TC68
0.3918  0.0993  -0.0587  -0.9856  0.3632  0.4144  13.5805  5.7218  -7.7957  6.3960
s.e.  0.0383  0.0418   0.0416   0.0142  0.0361  0.0341   1.3201  1.2980   1.3041  1.2763
AO80   AO121   TC229   TC270   AO416     AO445   TC634   AO780
-23.3707  5.5352  5.8088  7.0446  7.9304  -23.6372  5.5475  6.7194
s.e.    1.2376  1.2307  1.2594  1.2640  1.2476    1.2393  1.2598  1.2353

sigma^2 estimated as 2.332:  log likelihood=-1482.63
AIC=3003.26   AICc=3004.23   BIC=3092.34

Outliers:
type ind   time coefhat   tstat
1    LS   3   1:03  13.581  10.287
2    AO  52   8:03   5.722   4.408
3    AO  53   8:04  -7.796  -5.978
4    TC  68  10:05   6.396   5.012
5    AO  80  12:03 -23.371 -18.883
6    AO 121  18:02   5.535   4.498
7    TC 229  33:05   5.809   4.612
8    TC 270  39:04   7.045   5.573
9    AO 416  60:03   7.930   6.356
10   AO 445  64:04 -23.637 -19.073
11   TC 634  91:04   5.547   4.404
12   AO 780 112:03   6.719   5.439


Some of these outliers are already taken care of since they're the holidays. I'm not sure how to handle the rest of the outliers when fitting the model and in the CV.

What is the best way to go about taking care of the outliers? I can reset the values of the training data where it's predicted as an outliers to the recommended value if it's not a holiday for fitting the model and then still calculate the MAPE off of the original data. However, there's a LS at index 3 so I'm not sure that would make sense for that.

• Post your dataset. Specify the country and the beginning date. Commented Apr 5, 2016 at 16:16