I'm working on a toy problem to try and get a better understanding for time series forecasting. I have a sample data set, which I'll include, that shows daily e-commerce sales from 2015 through Feb, 2017.
I tried a arima model and holtwinter and they both show up as significant against my box test and they have residuals that are roughly normal and centered on zero.
But they give pretty wildly different predictions for the future. I'm curious if there are more ways I can evaluate to find out which model is correct. I'll include my data, my code and the forecasts.
Data
print(ts)
Time Series:
Start = c(2015, 1)
End = c(2017, 54)
Frequency = 365
[1] 2419.79487 3505.31069 3639.80925 1612.65730 2422.11190
[6] 6313.53477 5356.30223 1948.21457 1226.27511 2619.86945
[11] 1356.79885 2615.20348 912.33441 2000.43040 1508.11609
[16] 931.46061 122.07641 330.40875 834.80850 2237.23233
[21] 1510.53502 665.06950 6863.78618 639.22052 89.02246
[26] 2754.30644 268.26101 3697.11548 1267.50463 2372.73719
[31] 829.36903 1824.97771 2194.93701 3646.11185 3759.12441
[36] 2525.68906 2331.31828 890.27223 1120.82170 1805.74774
[41] 972.69618 2447.54541 3081.15920 1488.90617 147.64338
[46] 4340.27688 2176.31350 4411.59523 1792.95687 3609.35958
[51] 2286.18995 2222.49398 3593.66190 3900.08497 1892.80978
[56] 2665.22172 4276.53924 50.44121 4067.94254 2628.02807
[61] 541.83367 1117.90369 3015.66985 1016.28566 2537.17051
[66] 516.67809 6386.00200 418.82217 2949.50880 1689.55913
[71] 2949.91583 2726.65076 2884.76752 2229.08619 48.13187
[76] 3326.91954 2956.76084 2010.63298 871.95161 496.76691
[81] 2921.75271 1086.45719 1281.92972 846.14622 964.69261
[86] 2931.91579 2793.25232 719.45558 400.91642 10346.59135
[91] 1023.74518 2013.82867 2817.03580 4265.75481 2620.27636
[96] 900.70305 5631.09486 251.03763 3102.22359 2581.32491
[101] 5568.12748 2580.34334 3528.18579 6455.47190 2508.56013
[106] 3205.39106 1925.11516 1016.13844 1186.39525 3516.92571
[111] 1379.72208 2398.55234 2803.17791 647.70648 4692.30235
[116] 3368.73469 1883.75659 9133.57446 1812.21848 1564.08438
[121] 4227.42480 8565.40062 1924.97101 2223.61876 4181.95294
[126] 1977.34741 3760.25765 2499.33240 5044.08621 4765.99922
[131] 4041.15915 3821.23957 2415.37833 1962.89834 7403.01233
[136] 3019.91717 305.22969 4845.42196 445.80997 3610.68729
[141] 1383.78085 2767.74932 1734.36058 3292.70695 6547.25405
[146] 2513.03910 5560.55729 2716.36603 6231.96621 1500.73467
[151] 1497.32254 3266.17682 2870.74022 1681.59394 1530.76419
[156] 3104.80550 4872.74537 2815.93168 6455.70635 6545.21991
[161] 5092.66683 8178.01542 12568.84644 6322.05778 1765.43631
[166] 6626.02960 1668.03191 3914.24293 4566.22668 8365.60121
[171] 3099.42183 5446.67106 3200.00943 1224.58919 1559.96960
[176] 2138.57382 1906.66929 2422.35368 3616.86548 3334.00640
[181] 4507.14149 6125.92448 8092.22990 3794.75678 2865.56637
[186] 2227.67440 5956.54937 4588.17040 2043.45609 5434.65938
[191] 6875.76103 2305.23861 4346.69014 1597.73418 1940.89045
[196] 4257.32500 4772.42652 3589.84997 5067.06486 4945.17905
[201] 6301.70996 4148.84441 3657.72890 2864.09213 9503.47545
[206] 1255.75135 1748.96309 6303.93657 3382.48993 5573.37107
[211] 5371.02390 7488.83602 6247.43076 1436.65204 2765.54182
[216] 4784.58157 2853.47190 6896.63376 7417.30945 3119.94501
[221] 5465.63318 9422.17774 2417.14581 4929.78458 3495.64228
[226] 10153.46685 1187.32065 4873.73448 8732.30121 1587.15222
[231] 4601.50141 7653.46015 4663.40680 4144.93244 2475.20900
[236] 8588.03985 3347.64382 6091.36366 3314.00860 4044.20706
[241] 2097.46376 6283.00718 8423.43238 3744.23710 6027.19343
[246] 6336.40387 9338.00781 7554.75618 2756.94885 5186.63034
[251] 2421.60522 6987.49500 1597.06331 6720.19045 2744.80513
[256] 3038.76154 8178.66984 3926.00886 5940.65434 5300.48768
[261] 8471.32821 8928.95587 4701.39813 10041.44358 1680.14400
[266] 3968.45973 4472.53240 13293.86963 3699.19469 1467.66668
[271] 2461.11422 5750.48134 1414.33112 11885.92798 6386.73945
[276] 3378.33576 1713.67842 8359.66591 4028.41458 1744.99820
[281] 5981.58892 11398.67841 7566.50843 3817.43666 4632.71624
[286] 5901.94714 4984.39088 8637.71053 6499.54514 4092.10800
[291] 3251.86070 9449.60580 1854.83255 3778.81510 1893.59039
[296] 6996.37762 2710.17601 11662.40925 11698.90749 3855.53297
[301] 3771.92696 7342.43176 5014.54362 10157.92248 6429.03165
[306] 3454.77741 3460.79925 5958.29986 1866.37881 5927.29513
[311] 3647.21311 3617.65112 8250.16291 5369.64612 5961.59530
[316] 6740.12253 11509.91538 7687.55905 3289.38129 10515.51580
[321] 2819.35782 8545.49493 4219.39841 2180.53320 5634.11421
[326] 2980.08294 8943.12231 5906.84697 3458.74814 5231.12158
[331] 10244.34633 6439.00793 4403.12888 7397.66225 5676.87007
[336] 6051.90782 6847.33061 4331.30326 9647.08626 7596.30480
[341] 11282.27388 3605.06924 3887.58673 10847.82180 2404.46687
[346] 5898.25409 7008.69696 5065.71015 4244.98757 5000.98470
[351] 4512.22093 18559.29707 6126.12144 3362.73204 9170.10520
[356] 4991.86323 11226.08310 3967.26723 9927.51505 3202.16174
[361] 9744.17375 17283.79430 4992.50816 5511.12493 6089.36367
[366] 13661.67186 4781.35550 9060.74923 7968.16525 4301.33955
[371] 7915.49338 4051.91004 15439.58373 2625.92301 4636.23597
[376] 17518.70264 1489.57577 3752.88701 7186.81737 18418.39117
[381] 5215.34041 14320.49173 10836.93392 7122.68583 3827.34641
[386] 9658.48586 11540.33199 4874.93156 5413.79996 6559.87721
[391] 6698.35115 7450.31618 5124.40833 11848.79921 10472.24870
[396] 8947.89722 8322.23170 5562.67212 6697.50212 10278.24219
[401] 19850.43405 5668.77950 6669.72378 13966.87880 7848.34134
[406] 10852.47518 14119.84491 19456.65210 7549.29552 7646.82791
[411] 13858.53778 4870.83920 7321.03738 10130.07088 22499.67895
[416] 12859.81724 4525.16104 13087.18152 4211.61067 5297.96578
[421] 9758.97568 13191.58188 10540.24401 6805.64653 11401.71238
[426] 4674.43318 4894.43519 4595.99790 18100.55951 6155.16499
[431] 7537.91264 10502.84843 7197.80368 4492.28798 10915.47578
[436] 12137.70002 6246.59295 10366.29153 19848.90448 4425.16107
[441] 6414.59828 18595.67532 14526.65009 3581.70104 10361.29231
[446] 23150.30732 1629.64036 7787.46025 7666.88010 14235.82678
[451] 7035.54010 11977.38129 11801.51601 12376.49013 5831.76278
[456] 9898.75818 16525.40683 5768.75332 10566.59566 24055.29488
[461] 10073.53801 9009.13923 9319.13438 13907.98684 10925.22464
[466] 9792.87148 16977.72332 3774.61189 8413.25801 9750.37538
[471] 28654.80504 7348.77271 8303.36030 12211.30824 9790.53693
[476] 9601.36392 9825.83614 22485.91512 11552.05697 5450.97418
[481] 19013.68849 7399.47044 10804.38369 6607.49470 8698.30219
[486] 9045.30062 14120.25877 13995.96400 12991.81683 8942.38113
[491] 6139.69811 10048.96559 9713.64216 8688.18051 27664.79188
[496] 8692.95157 6988.05436 12573.54472 13313.57444 10093.68181
[501] 3542.07225 24604.21248 10821.76302 5245.51342 11803.05721
[506] 9626.88337 13010.85279 16794.29634 16541.29423 7811.71742
[511] 11497.11999 9490.90880 15691.86496 11322.97888 12059.74836
[516] 29516.08831 8788.84032 10553.92136 13731.43889 16354.18013
[521] 17382.61772 10376.21927 21303.19616 6331.06522 6978.52080
[526] 9406.21650 21248.21329 19079.57078 16643.10817 14178.88970
[531] 9777.38379 10307.55443 9122.59614 23274.43110 17496.65486
[536] 7988.07085 19218.40662 10822.65428 10373.89506 10572.60838
[541] 19175.31931 18464.68286 11610.55153 26854.05031 9342.41123
[546] 14780.56518 18678.74688 12739.95370 11753.04279 5443.18343
[551] 27418.34257 10140.25222 15708.84392 17922.23717 25052.15779
[556] 13128.82365 9235.30723 23451.62767 10084.03463 10317.88871
[561] 14292.19289 27673.55812 10063.59796 12965.26029 9704.30161
[566] 9460.62887 11391.32989 10939.61661 22451.54530 9323.79815
[571] 15173.53856 16338.60835 8907.35576 10319.21918 15804.69749
[576] 22248.90622 10582.54606 11571.06553 22203.13622 13314.77408
[581] 20050.21434 15310.16295 13890.42942 11648.31834 6440.00580
[586] 18626.58934 6865.56176 10249.83446 16047.41271 21852.21472
[591] 22609.51047 9338.29901 12546.99784 7509.13902 11543.12718
[596] 14926.37920 29941.14068 16301.91987 13267.19478 20968.55969
[601] 9888.97530 11325.35852 14439.56179 31678.69639 15126.80778
[606] 11677.04261 19254.75438 12989.73702 14566.32013 15678.70193
[611] 14385.02343 12792.98629 16176.70584 33500.16830 13180.92931
[616] 16400.46322 29907.77121 21393.57842 15766.37736 12175.14383
[621] 19742.19402 12480.24253 9667.78266 19113.82963 27009.60011
[626] 17181.73216 12862.94088 20154.16806 8918.23393 10656.65783
[631] 15354.68490 35984.08773 21859.95243 12571.96797 24392.90865
[636] 13523.15454 17531.80812 17107.54903 16011.70841 19153.31569
[641] 13507.97139 24859.11373 12160.92744 7658.77275 12652.25339
[646] 20786.80630 19219.05594 14001.75736 23930.09355 14781.50495
[651] 16768.75157 17523.10956 27227.61232 16986.16377 11866.47244
[656] 32843.62019 10429.48463 19272.35178 32215.45883 25017.83671
[661] 11263.33090 16578.51746 35580.88951 14912.92940 16244.67581
[666] 19365.00181 30946.50694 18750.29772 17461.82259 24688.60349
[671] 13733.52349 11490.22267 22605.09096 19785.42353 19251.76745
[676] 15444.40320 26905.76718 11515.96307 17475.84202 20306.29457
[681] 26201.40881 23415.37499 13463.03016 29751.80143 14979.88358
[686] 19665.15285 24472.71512 33866.71294 23968.12308 14954.76833
[691] 36554.00655 19123.46527 15219.26998 33079.20749 23262.99565
[696] 19912.45876 13201.47413 41280.95619 15334.89906 21605.91294
[701] 16903.39509 37435.43722 21108.17089 23374.52077 35857.52232
[706] 21393.97474 21250.53593 24892.08621 24498.68640 18589.12106
[711] 19965.10975 36869.90623 12690.09279 14958.43143 21128.64867
[716] 28153.35056 15406.95845 15033.53285 43978.06743 15926.07385
[721] 17957.29328 27003.36373 32398.15296 26932.84454 19612.79280
[726] 27972.24553 13460.94492 15299.55838 23859.95146 26053.24987
[731] 18806.59707 19577.55827 51509.19952 16139.87143 19866.36628
[736] 18413.60302 32198.78960 21738.99945 18004.37841 38202.37585
[741] 15216.14492 18578.28043 26806.46957 30772.19407 18713.38384
[746] 18692.23514 41750.56528 14110.49392 16397.12035 19966.07880
[751] 26811.19010 22338.63547 22835.77244 30527.05657 13049.25827
[756] 12460.07663 30202.84367 30332.39829 24582.66513 16044.06816
[761] 36128.65052 19173.09364 17540.32332 15495.95355 39799.96467
[766] 18238.17887 19218.93387 40014.99026 14646.82376 18970.33616
[771] 29382.22482 28078.51323 24561.64152 20083.82957 38732.65776
[776] 18797.43935 18347.29107 23013.29072 19419.47252 18180.54027
[781] 17270.98478 41243.08957 17319.42314 17030.13515
Code
library(forecast)
# model
# arima
find_arima <- auto.arima(ts, stepwise = FALSE, approximation = FALSE)
fit <- arima(ts, c(4, 1, 1))
plot(forecast(find_arima, h = 365))
# holtwinter
hw <- HoltWinters(ts)
hw_forecast <- forecast(hw, h = 365)
# evaluate
# arima
acf(find_arima$residuals, lag.max=200)
Box.test(find_arima$residuals, lag=200, type="Ljung-Box")
plot(find_arima$residuals)
# holtwinter
acf(hw_forecast, lag.max=20)
Box.test(hw_forecast$residuals, lag=200, type="Ljung-Box")
plot(hw_forecast$residuals)
Forecasts
Next Year Forecasts with ARIMA
Next Year Forecasts with HoltWinter
ARIMA shows a consistent positive trend for the next year while HoltWinter indicates it could go in either direction, but it will probably be negative.
I know in a regular model I can build a test set and compare against that, but I don't know if that holds true with time series models. Is there a definitive way to find out which model is better?
I split my data into train and test and this is the forecast with the lowest RMSE.
