2
$\begingroup$

I've tried ets, tbats, and arima - I can't seem to get anything but a straight line out of this.

Example I tried:

hbf_ts2 <- msts(hbf$TheValue, seasonal.periods = c(7, 365.25))
fit2 <- tbats(hbf_ts2)
fc2 <- forecast(fit2, h=60)
plot(fc2)

Putting this into the PowerBI forecaster though yielded a meaningful forecast with no problem (2 mins, 2 maxes at the appropriate times of the forecasted year).

Edit: Added PowerBI pic showing what I believe are correct mins around mar/apr & sep/oct and maxes around jul/aug & dec. My question can be interpreted as: how to reverse engineer what PowerBI sees? enter image description here

Can somebody show me how to get a forecast to pick out the pattern(s) in this data?

    TheDate TheValue
2016-01-01  492905
2016-01-02  1601949
2016-01-03  1151405
2016-01-04  2186586
2016-01-05  1113354
2016-01-06  309563
2016-01-07  2055883
2016-01-08  999319
2016-01-09  2354847
2016-01-10  971635
2016-01-11  711481
2016-01-12  1187269
2016-01-13  1493296
2016-01-14  1656428
2016-01-15  1372378
2016-01-16  1649070
2016-01-17  1372722
2016-01-18  1066177
2016-01-19  2289296
2016-01-20  463604
2016-01-21  1783337
2016-01-22  591826
2016-01-23  1921277
2016-01-24  1002645
2016-01-25  1288591
2016-01-26  1801679
2016-01-27  2463919
2016-01-28  1145362
2016-01-29  963447
2016-01-30  1722081
2016-01-31  1112340
2016-02-01  1389396
2016-02-02  1018990
2016-02-03  1290466
2016-02-04  1274589
2016-02-05  1027891
2016-02-06  2120179
2016-02-07  1766689
2016-02-08  1159126
2016-02-09  2731442
2016-02-10  1380733
2016-02-11  957275
2016-02-12  692530
2016-02-13  926444
2016-02-14  500533
2016-02-15  783966
2016-02-16  1045921
2016-02-17  1807967
2016-02-18  2594422
2016-02-19  434663
2016-02-20  2057508
2016-02-21  1991184
2016-02-22  1134781
2016-02-23  1169244
2016-02-24  1315783
2016-02-25  505407
2016-02-26  697467
2016-02-27  930899
2016-02-28  836639
2016-02-29  1175764
2016-03-01  1729977
2016-03-02  1214509
2016-03-03  1313172
2016-03-04  1898133
2016-03-05  567347
2016-03-06  570458
2016-03-07  797164
2016-03-08  262597
2016-03-09  1133707
2016-03-10  1474149
2016-03-11  993599
2016-03-12  1140452
2016-03-13  470952
2016-03-14  2144962
2016-03-15  1010312
2016-03-16  816210
2016-03-17  778302
2016-03-18  1410789
2016-03-19  2098186
2016-03-20  617023
2016-03-21  783786
2016-03-22  984688
2016-03-23  896679
2016-03-24  802999
2016-03-25  992319
2016-03-26  803603
2016-03-27  412898
2016-03-28  1041051
2016-03-29  1203917
2016-03-30  609461
2016-03-31  1277114
2016-04-01  1513692
2016-04-04  1615950
2016-04-05  2399861
2016-04-06  1568040
2016-04-07  1785726
2016-04-08  993752
2016-04-09  1183265
2016-04-10  1186096
2016-04-11  842837
2016-04-12  1673087
2016-04-13  1476926
2016-04-14  1958557
2016-04-15  418363
2016-04-16  592586
2016-04-18  418754
2016-04-19  1733697
2016-04-20  1304147
2016-04-21  1045310
2016-04-22  860830
2016-04-23  1840458
2016-04-24  744558
2016-04-25  1046460
2016-04-26  1246456
2016-04-27  714769
2016-04-28  1595069
2016-04-29  764510
2016-04-30  1943821
2016-05-01  1345685
2016-05-02  860365
2016-05-03  1582654
2016-05-04  1159752
2016-05-05  911923
2016-05-06  954731
2016-05-07  860921
2016-05-08  2082131
2016-05-09  2401106
2016-05-10  1586094
2016-05-11  1513561
2016-05-12  551191
2016-05-13  942977
2016-05-14  1514368
2016-05-15  834673
2016-05-16  1464914
2016-05-17  2825643
2016-05-18  1919046
2016-05-19  1106938
2016-05-20  1477300
2016-05-21  1389177
2016-05-22  1131176
2016-05-23  1013731
2016-05-24  1770357
2016-05-25  1346478
2016-05-26  1302532
2016-05-27  2240548
2016-05-28  1653050
2016-05-29  1969550
2016-05-30  797389
2016-05-31  1979795
2016-06-01  1020901
2016-06-02  1494291
2016-06-03  1976515
2016-06-04  1905873
2016-06-05  1303286
2016-06-06  942723
2016-06-07  2214164
2016-06-08  2321545
2016-06-09  1177346
2016-06-10  1240553
2016-06-11  1458808
2016-06-12  874046
2016-06-13  1423719
2016-06-14  1368339
2016-06-15  1472794
2016-06-16  1681757
2016-06-17  1191515
2016-06-18  1794076
2016-06-19  1338900
2016-06-20  2128570
2016-06-21  2709468
2016-06-22  1481802
2016-06-23  1820330
2016-06-24  1775269
2016-06-25  1350577
2016-06-26  1768682
2016-06-27  1489650
2016-06-28  2198414
2016-06-29  1106565
2016-06-30  2088400
2016-07-01  1983991
2016-07-02  1377339
2016-07-03  1817443
2016-07-04  1135847
2016-07-05  2552720
2016-07-06  1105784
2016-07-07  1807094
2016-07-08  1524917
2016-07-09  652983
2016-07-10  576085
2016-07-11  1568422
2016-07-12  2221462
2016-07-13  2717538
2016-07-14  1824117
2016-07-15  2261913
2016-07-16  1572669
2016-07-17  2366064
2016-07-18  2385405
2016-07-19  1931224
2016-07-20  1724003
2016-07-21  2035054
2016-07-22  1235362
2016-07-23  1308190
2016-07-24  1986094
2016-07-25  1958964
2016-07-26  1691101
2016-07-27  2205191
2016-07-28  1051101
2016-07-29  757187
2016-07-30  1751114
2016-07-31  1071575
2016-08-01  2059648
2016-08-02  2116452
2016-08-03  2225879
2016-08-04  3102684
2016-08-05  1290212
2016-08-06  1479507
2016-08-07  1494233
2016-08-08  2292587
2016-08-09  1527557
2016-08-10  1401083
2016-08-11  1139128
2016-08-12  1322069
2016-08-13  1659380
2016-08-14  1260246
2016-08-15  1930342
2016-08-16  1002820
2016-08-17  2002754
2016-08-18  1429877
2016-08-19  1505114
2016-08-20  2547243
2016-08-21  1473001
2016-08-22  1013516
2016-08-23  1626099
2016-08-24  2327920
2016-08-25  1989945
2016-08-26  1863659
2016-08-27  1051880
2016-08-28  1995427
2016-08-29  1096118
2016-08-30  2380033
2016-08-31  2267014
2016-09-01  1015147
2016-09-02  821369
2016-09-03  750505
2016-09-06  1750191
2016-09-07  2343824
2016-09-08  1132369
2016-09-09  490321
2016-09-10  886485
2016-09-11  1348620
2016-09-12  1278910
2016-09-13  1196608
2016-09-14  1256872
2016-09-15  1441281
2016-09-16  795341
2016-09-17  1478755
2016-09-18  2116274
2016-09-19  697804
2016-09-20  1743386
2016-09-21  240823
2016-09-29  1027833
2016-09-30  1286467
2016-10-01  901308
2016-10-02  693694
2016-10-03  2326552
2016-10-04  1681307
2016-10-05  1677090
2016-10-06  1323572
2016-10-07  874972
2016-10-08  1128333
2016-10-09  1129885
2016-10-10  1881623
2016-10-11  1239745
2016-10-12  1507595
2016-10-13  1373724
2016-10-14  147352
2016-10-17  1027790
2016-10-18  1571171
2016-10-19  1789737
2016-10-20  1485559
2016-10-21  989748
2016-10-22  1643482
2016-10-23  830737
2016-10-24  2306988
2016-10-25  2319400
2016-10-26  4096735
2016-10-27  1333566
2016-10-28  1190304
2016-10-29  1220414
2016-10-30  829467
2016-10-31  1186065
2016-11-01  328606
2016-11-02  1652422
2016-11-03  1080530
2016-11-04  1377598
2016-11-05  3303917
2016-11-06  1314589
2016-11-07  1320031
2016-11-08  1789069
2016-11-09  648243
2016-11-10  1530916
2016-11-11  2132679
2016-11-12  1509925
2016-11-13  2477339
2016-11-14  1834884
2016-11-15  1225178
2016-11-16  455581
2016-11-17  1291606
2016-11-18  1031542
2016-11-19  1168160
2016-11-20  1817138
2016-11-21  1572821
2016-11-22  2879111
2016-11-23  1348398
2016-11-24  473761
2016-11-25  1114251
2016-11-26  2321607
2016-11-27  1246556
2016-11-28  766522
2016-11-29  694074
2016-11-30  822716
2016-12-01  875663
2016-12-02  1795062
2016-12-03  1880241
2016-12-04  763947
2016-12-05  1208156
2016-12-06  1354850
2016-12-07  1211518
2016-12-08  797432
2016-12-09  1303963
2016-12-10  1477425
2016-12-11  874991
2016-12-12  639003
2016-12-14  1079014
2016-12-15  1319408
2016-12-16  612839
2016-12-17  1872374
2016-12-18  717536
2016-12-19  1532533
2016-12-20  1524805
2016-12-21  1975479
2016-12-22  1506133
2016-12-23  994251
2016-12-24  1520409
2016-12-26  2619131
2016-12-27  2338902
2016-12-28  1848276
2016-12-29  1038538
2016-12-30  1403629
2016-12-31  1024591
2017-01-01  911706
2017-01-02  1018040
2017-01-03  2912521
2017-01-04  902741
2017-01-05  2171646
2017-01-06  885490
2017-01-07  3252669
2017-01-08  431399
2017-01-09  2396805
2017-01-11  1091594
2017-01-12  869960
2017-01-13  2653667
2017-01-14  1206634
2017-01-15  1351949
2017-01-16  2613909
2017-01-17  1050561
2017-01-18  1249898
2017-01-19  861566
2017-01-20  1057132
2017-01-21  1854328
2017-01-22  1748786
2017-01-23  629802
2017-01-24  1367775
2017-01-25  1111216
2017-01-26  1949142
2017-01-27  819318
2017-01-28  1839211
2017-01-29  175179
2017-01-30  800008
2017-01-31  1407335
2017-02-01  1038374
2017-02-02  669512
2017-02-03  778464
2017-02-04  1457364
2017-02-05  797369
2017-02-06  1118268
2017-02-07  1276264
2017-02-08  1578296
2017-02-09  2960152
2017-02-10  1143752
2017-02-11  1958829
2017-02-12  3245721
2017-02-13  1356475
2017-02-14  1062092
2017-02-15  881009
2017-02-16  1266027
2017-02-17  829273
2017-02-18  2103150
2017-02-19  1335047
2017-02-20  2437420
2017-02-22  974614
2017-02-23  1189815
2017-02-24  1012674
2017-02-25  1828608
2017-02-26  1655255
2017-02-27  1173650
2017-02-28  504155
2017-03-01  730735
2017-03-02  1596667
2017-03-03  458890
2017-03-04  2297390
2017-03-05  1655067
2017-03-06  2227014
2017-03-07  1950607
2017-03-08  1933252
2017-03-09  738067
2017-03-10  1119581
2017-03-11  1166814
2017-03-12  820901
2017-03-13  1526354
2017-03-14  1165152
2017-03-15  1736698
2017-03-16  2289070
2017-03-17  1407411
2017-03-18  2582293
2017-03-19  1661416
2017-03-20  852530
2017-03-21  1984492
2017-03-22  1222604
2017-03-23  683434
2017-03-24  2359227
2017-03-25  1566197
2017-03-26  1294745
2017-03-27  1174317
2017-03-28  1118732
2017-03-29  1234170
2017-03-30  731860
2017-03-31  1909236
2017-04-01  920438
2017-04-02  2489688
2017-04-03  435190
2017-04-04  415410
2017-04-06  1060136
2017-04-07  1329207
2017-04-08  1164398
2017-04-09  891000
2017-04-10  988360
2017-04-11  1544656
2017-04-12  1634726
2017-04-13  1464269
2017-04-14  1604310
2017-04-15  1184282
2017-04-16  571598
2017-04-17  1948306
2017-04-18  1361531
2017-04-19  884592
2017-04-20  1534833
2017-04-21  1281832
2017-04-22  1544057
2017-04-23  1536583
2017-04-24  694634
2017-04-25  2178288
2017-04-26  264247
2017-04-27  23204
2017-04-28  1855550
2017-04-29  1251140
2017-04-30  1713568
2017-05-01  1207142
2017-05-02  996342
2017-05-03  1658149
2017-05-04  1846606
2017-05-05  1360394
2017-05-06  1281634
2017-05-07  1615912
2017-05-08  876281
2017-05-09  1362545
2017-05-10  1613172
2017-05-11  1036849
2017-05-12  1074509
2017-05-13  2008791
2017-05-14  1006338
2017-05-15  960582
2017-05-16  993703
2017-05-17  1196713
2017-05-18  521948
2017-05-19  1043909
2017-05-20  1400979
2017-05-21  868392
2017-05-22  1350697
2017-05-23  633453
2017-05-24  1718490
2017-05-25  1257125
2017-05-26  1363557
2017-05-27  1851730
2017-05-28  1622941
2017-05-29  1551901
2017-05-30  1195573
2017-05-31  1520805
2017-06-01  1102248
2017-06-02  1028280
2017-06-03  1287205
2017-06-04  1062718
2017-06-05  1407302
2017-06-06  1748039
2017-06-07  1888155
2017-06-08  818205
2017-06-09  1094362
2017-06-10  2025510
2017-06-11  1201594
2017-06-12  2413791
2017-06-13  1567467
2017-06-14  1259972
2017-06-15  3053001
2017-06-16  1629186
2017-06-17  1489205
2017-06-18  175168
2017-06-19  1276806
2017-06-20  2804043
2017-06-21  1674382
2017-06-22  722361
2017-06-23  747358
2017-06-24  2219739
2017-06-25  1293540
2017-06-26  1545274
2017-06-27  2593252
2017-06-28  1253544
2017-06-29  1881070
2017-06-30  2713208
2017-07-01  1302957
2017-07-02  1517749
2017-07-03  1664217
2017-07-04  1491774
2017-07-05  1501322
2017-07-06  1609092
2017-07-07  1510391
2017-07-08  2176957
2017-07-09  1867961
2017-07-10  1742991
2017-07-11  2013235
2017-07-12  1539111
2017-07-13  2366031
2017-07-14  1751373
2017-07-15  1539263
2017-07-16  1504272
2017-07-17  1704127
2017-07-18  1852088
2017-07-19  1403717
2017-07-20  1329328
2017-07-21  628820
2017-07-22  946526
2017-07-23  972992
2017-07-24  1027360
2017-07-25  1665103
2017-07-26  1999804
2017-07-27  1394441
2017-07-29  3453899
2017-07-30  1208764
2017-07-31  1130972
2017-08-01  2458168
2017-08-02  1788473
2017-08-03  1747692
2017-08-04  1167481
2017-08-05  1428286
2017-08-06  2095607
2017-08-07  1674846
2017-08-08  2610214
2017-08-09  1511836
2017-08-10  3123539
2017-08-11  2791303
2017-08-12  3569525
2017-08-13  226607
2017-08-14  1694048
2017-08-15  2575182
2017-08-16  1618226
2017-08-17  1859182
2017-08-18  1366536
2017-08-19  3014336
2017-08-20  2015712
2017-08-21  1861626
2017-08-22  2121916
2017-08-23  2022347
2017-08-24  1387513
2017-08-25  2106167
2017-08-26  1872918
2017-08-27  1626310
2017-08-28  1376909
2017-08-29  1430963
2017-08-30  2235554
2017-08-31  1081856
2017-09-01  1000952
2017-09-02  1487497
2017-09-03  1910583
2017-09-04  732854
2017-09-05  2319999
2017-09-06  1945352
2017-09-07  1389082
2017-09-08  1563222
2017-09-09  3204633
2017-09-10  2438413
2017-09-11  1675811
2017-09-12  1406697
2017-09-13  628640
2017-09-14  1624733
2017-09-15  571868
2017-09-16  1962417
2017-09-17  1078254
2017-09-18  2305921
2017-09-19  2112974
2017-09-20  2075621
2017-09-21  1559805
2017-09-22  1927532
2017-09-23  1608380
2017-09-24  793252
2017-09-25  1529915
2017-09-26  715803
2017-09-27  804610
2017-09-28  775891
2017-09-29  1507243
2017-09-30  1455677
2017-10-01  1078208
2017-10-02  409310
2017-10-04  11587
2017-10-05  1557906
2017-10-06  517875
2017-10-07  1163345
2017-10-08  3201790
2017-10-09  2217903
2017-10-10  1634903
2017-10-11  2815781
2017-10-12  2003950
2017-10-13  808803
2017-10-14  1243344
2017-10-15  1328222
2017-10-16  1125323
2017-10-17  1050608
2017-10-18  934657
2017-10-19  1296011
2017-10-20  2480450
2017-10-21  501921
2017-10-22  1550343
2017-10-23  1419114
2017-10-24  1166513
2017-10-25  2230667
2017-10-26  1310241
2017-10-27  976972
2017-10-28  1836926
2017-10-29  915576
2017-10-30  988285
2017-10-31  1106031
2017-11-01  1765272
2017-11-02  1245311
2017-11-03  708948
2017-11-04  1348570
2017-11-05  1509031
2017-11-06  1257916
2017-11-07  1125341
2017-11-08  2043454
2017-11-09  1128279
2017-11-10  1300628
2017-11-11  1572715
2017-11-12  1119639
2017-11-13  913999
2017-11-14  922766
2017-11-15  1617464
2017-11-16  901141
2017-11-17  472597
2017-11-18  2133865
2017-11-19  1466001
2017-11-20  3713672
2017-11-21  1486944
2017-11-22  1341622
2017-11-23  329239
2017-11-24  1499170
2017-11-25  1696232
2017-11-26  1661195
2017-11-27  1229017
2017-11-28  1406481
2017-11-29  926347
2017-11-30  325500
2017-12-04  1780190
2017-12-05  1349706
2017-12-06  1195912
2017-12-07  2023148
2017-12-08  563024
2017-12-09  1364205
2017-12-10  1581158
2017-12-11  2523115
2017-12-12  1922615
2017-12-13  2469427
2017-12-14  2820303
2017-12-15  664505
2017-12-16  3051759
2017-12-17  2787198
2017-12-18  1330068
2017-12-19  3655284
2017-12-20  2582808
2017-12-21  3029379
2017-12-22  1492587
2017-12-23  1587996
2017-12-24  717386
2017-12-26  1652614
2017-12-27  2254227
2017-12-28  1997810
2017-12-29  1023016
2017-12-30  2083158
2017-12-31  3360735
2018-01-01  2291110
2018-01-02  1211079
2018-01-03  1954208
2018-01-04  1090845
2018-01-05  1613957
2018-01-06  2831025
2018-01-07  1339702
2018-01-08  1461118
2018-01-09  1612776
2018-01-10  920066
2018-01-11  1554299
2018-01-12  1327704
2018-01-13  2105399
2018-01-14  856336
2018-01-15  1418207
2018-01-16  1563512
2018-01-17  1677243
2018-01-18  1549844
2018-01-19  1631164
2018-01-20  1653606
2018-01-21  936179
2018-01-22  1393694
2018-01-23  1244791
2018-01-24  628210
2018-01-25  1993233
2018-01-26  1729333
2018-01-27  3975304
2018-01-28  336512
2018-01-29  2108933
2018-01-30  1782751
2018-01-31  1551629
2018-02-01  1243868
2018-02-02  609144
2018-02-03  1850660
2018-02-04  3179741
2018-02-05  1055656
2018-02-06  507421
2018-02-07  2234207
2018-02-08  792326
2018-02-09  283808
2018-02-10  867622
2018-02-11  1177006
2018-02-12  855364
2018-02-13  365648
2018-02-14  1839964
2018-02-15  95938
2018-02-16  1093658
2018-02-17  503150
2018-02-18  1547753
2018-02-19  1341982
2018-02-20  1145113
2018-02-21  1071130
2018-02-22  1530749
2018-02-25  1112171
2018-02-26  1020162
2018-02-27  1416732
2018-02-28  827852
2018-03-01  1285696
2018-03-02  1581333
2018-03-03  2177187
2018-03-04  340061
2018-03-06  1540062
2018-03-07  1000615
2018-03-08  1017100
2018-03-09  1630180
2018-03-10  1175505
2018-03-11  2199139
2018-03-12  1141769
2018-03-13  463070
2018-03-14  1360353
2018-03-15  927530
2018-03-16  1520612
2018-03-17  1461301
2018-03-18  1665564
2018-03-19  1214638
2018-03-20  479268
2018-03-21  1183624
2018-03-22  617556
2018-03-23  2110457
2018-03-24  1161841
2018-03-25  2370691
2018-03-26  1647063
2018-03-27  812469
2018-03-28  1000926
2018-03-29  1946355
2018-03-30  2148340
2018-03-31  2003400
2018-04-01  945284
2018-04-02  1815191
2018-04-03  937982
2018-04-04  811794
2018-04-05  1182048
2018-04-06  973016
2018-04-07  1871696
2018-04-08  1165730
2018-04-09  1581681
2018-04-10  1301859
2018-04-11  1068074
2018-04-12  960951
2018-04-13  1964118
2018-04-14  1358190
2018-04-15  937033
2018-04-16  1178911
2018-04-17  1149829
2018-04-18  1193448
2018-04-19  982415
2018-04-20  1916077
2018-04-21  1631718
2018-04-22  2066066
2018-04-23  940597
2018-04-24  2202644
2018-04-25  1141160
2018-04-26  976945
2018-04-27  2350759
2018-04-28  4797161
2018-05-07  2267920
2018-05-08  1760160
2018-05-09  1796219
2018-05-10  1863167
2018-05-11  1926965
2018-05-12  1794331
2018-05-13  664100
2018-05-14  1442932
2018-05-15  2488486
2018-05-16  1589911
2018-05-17  1491023
2018-05-18  1112915
2018-05-19  1645927
2018-05-20  1989534
2018-05-21  1762019
2018-05-22  2506825
2018-05-23  3075061
2018-05-24  474683
2018-05-25  2451915
2018-05-26  820202
2018-05-27  1515221
2018-05-28  2165594
2018-05-29  3296624
2018-05-30  220517
2018-05-31  1181929
2018-06-01  775327
2018-06-02  2132940
2018-06-03  1005578
2018-06-04  1857927
2018-06-05  2710759
2018-06-06  2151611
2018-06-07  590602
2018-06-09  994103
2018-06-10  1189664
2018-06-11  1804087
2018-06-12  3249893
2018-06-13  2140094
2018-06-14  4001751
2018-06-15  952523
2018-06-16  1004758
2018-06-17  765776
2018-06-18  1928883
2018-06-19  2353313
2018-06-20  781372
2018-06-21  1442026
2018-06-22  678432
2018-06-23  2885504
2018-06-24  819634
2018-06-25  1863066
2018-06-26  1328277
2018-06-27  2221043
2018-06-28  1522942
2018-06-29  1896754
2018-06-30  1196607
2018-07-01  546782
2018-07-02  2179502
2018-07-03  1570718
2018-07-04  3333997
2018-07-05  2702911
2018-07-06  1272268
2018-07-07  1855696
2018-07-08  1783158
2018-07-09  1444503
2018-07-10  2053629
2018-07-11  906335
2018-07-12  1864814
2018-07-13  717539
2018-07-14  2136551
2018-07-15  1273029
2018-07-16  1772809
2018-07-17  1661216
2018-07-18  2713665
2018-07-19  1363139
2018-07-20  1141465
2018-07-21  1253139
2018-07-22  1331203
2018-07-23  1491406
2018-07-24  1402238
2018-07-25  75357
2018-07-26  1525053
2018-07-27  2397020
2018-07-28  1971354
2018-07-29  1772911
2018-07-30  3052149
2018-07-31  2361227
2018-08-01  1360559
2018-08-02  2519580
2018-08-03  988896
2018-08-04  2677285
2018-08-05  1171423
2018-08-06  2155847
2018-08-07  1461634
2018-08-08  1452754
2018-08-09  1437879
2018-08-10  1132824
2018-08-11  982006
2018-08-12  1120162
2018-08-13  1671429
2018-08-14  3025344
2018-08-15  1069422
2018-08-16  1195537
2018-08-17  882980
2018-08-18  2760260
2018-08-19  1459098
2018-08-20  1574928
2018-08-21  1842542
2018-08-22  1857758
2018-08-23  1811214
2018-08-24  1177500
2018-08-25  2351327
2018-08-26  1253109
2018-08-27  1352088
2018-08-28  2468555
2018-08-29  1872385
2018-08-30  1391215
2018-08-31  1378033
2018-09-01  1153951
2018-09-02  901030
2018-09-03  1949716
2018-09-04  1754018
2018-09-05  2064959
2018-09-06  437698
2018-09-07  953701
2018-09-08  602979
2018-09-09  737062
2018-09-10  1466196
2018-09-11  1316563
2018-09-12  1419652
2018-09-13  1513515
2018-09-14  1100097
2018-09-15  2513661
2018-09-16  1338282
2018-09-17  1282230
2018-09-18  1621023
2018-09-19  712422
2018-09-20  1069738
2018-09-21  1093389
2018-09-22  1757139
2018-09-23  2229094
2018-09-24  1608801
2018-09-25  1776676
2018-09-26  715061
2018-09-27  2061564
2018-09-28  1775204
2018-09-29  596271
2018-09-30  1052694
2018-10-01  2005555
2018-10-02  1201587
2018-10-03  1506207
2018-10-04  1135962
2018-10-05  1425691
2018-10-06  1929594
2018-10-07  1558859
2018-10-08  2644698
2018-10-09  2168196
2018-10-10  601651
2018-10-11  704045
2018-10-12  1468953
2018-10-13  1144636
2018-10-14  621194
2018-10-15  1507004
2018-10-16  1328006
2018-10-17  905645
2018-10-18  946435
2018-10-19  1379612
2018-10-20  1165128
2018-10-21  1662357
2018-10-22  1934873
2018-10-23  2031270
2018-10-24  1404769
2018-10-25  1107453
2018-10-26  1065420
2018-10-27  1436357
2018-10-28  1283520
2018-10-29  687467
2018-10-30  1542343
2018-10-31  1857454
2018-11-01  1348635
2018-11-02  971220
2018-11-03  1579107
2018-11-05  1850613
2018-11-06  2203924
2018-11-07  1478081
2018-11-08  904263
2018-11-09  894744
2018-11-10  2138165
2018-11-11  2545984
2018-11-12  1273497
2018-11-13  1234634
2018-11-14  1319511
2018-11-15  750258
2018-11-16  1607310
2018-11-17  595905
2018-11-18  1045006
2018-11-19  3356163
2018-11-20  1962632
2018-11-21  854751
2018-11-22  295059
2018-11-23  906795
2018-11-24  713094
2018-11-25  1043280
2018-11-26  974499
2018-11-27  1200869
2018-11-28  1252872
2018-11-29  1189154
2018-11-30  1647594
2018-12-01  1743667
2018-12-03  339700
2018-12-04  1814194
2018-12-05  1140506
2018-12-06  2009347
2018-12-07  1304115
2018-12-08  1986904
2018-12-09  1392328
2018-12-10  1466535
2018-12-11  1634210
2018-12-12  1201033
2018-12-13  1518156
2018-12-14  911461
2018-12-15  2941987
2018-12-16  1313848
2018-12-17  1958368
2018-12-18  1606970
2018-12-19  2916651
2018-12-20  1509846
2018-12-21  938201
2018-12-22  1204572
2018-12-23  2729989
2018-12-24  2192122
2018-12-26  2516399
2018-12-27  1668535
2018-12-30  1600417
2018-12-31  2237816
$\endgroup$
  • 3
    $\begingroup$ There is very little pattern I can discern in your data. Where are the "appropriate times" at which you believe there should be a maximum or a minimum? A flat line can be surprisingly often the optimal forecast, so have you compared the performance of PowerBI and say msts() on a holdout sample? I may write up an answer later on why I see very little pattern here. $\endgroup$ – Stephan Kolassa Jan 22 at 7:40
  • $\begingroup$ Updated question with the PowerBI comparison. $\endgroup$ – sherifffruitfly Jan 22 at 9:56
  • $\begingroup$ Thank you. This is actually a very good question. I will write up an answer later today or tomorrow, and I ask the close and down voters to hold back. $\endgroup$ – Stephan Kolassa Jan 22 at 10:02
  • 2
    $\begingroup$ Yes, this question asks how to perform an action in R. However, it needs statistical expertise to understand or answer, which is a direct quote from the page detailing what is on topic here. Specifically, I intend to argue why a flat line forecast is probably the most appropriate here for statistical reasons, and why PowerBI is likely peddling snake oil. I have nominated the question for reopening and hope for the best. $\endgroup$ – Stephan Kolassa Jan 22 at 16:42
2
$\begingroup$

You have three years of daily data, which is a good amount. As you write, you may have (intra-weekly and intra-yearly), so it is a good idea to look at models that can model such patterns, like or .

Let's start by looking at your series (I'm using R, and I created a data.frame similar to your hbf):

with(hbf,plot(TheDate,TheValue,type="o",pch=19))

series

No patterns are readily apparent. Let us use appropriate seasonplots to examine any seasonalities. We will start with the intra-weekly seasonality:

library(forecast)
seasonplot(ts(hbf$TheValue,frequency=7),pch=19)

seasonplot_weekly

I do not see a seasonal pattern here. Let's look at an STL decomposition:

plot(stl(ts(hbf$TheValue,frequency=7),s.window="periodic"))

STL_weekly

Again, there is no visible seasonality here. (Yes, there is a seasonal component in the STL plot, but note the scales given by the gray boxes. The seasonal component is completely negligible. See here for more information.)

For good measure, here are beanplots of your series by weekday:

library(beanplot)
with(hbf,beanplot(TheValue~TheDateFactor,what=c(0,1,0,0),col="lightgray",border=NA))
with(hbf,points(as.numeric(TheDateFactor)+runif(nrow(hbf),-.3,.3),TheValue,pch=19,cex=0.6))

beanplot_weekly

Again, there is no weekly pattern.

We can repeat the exercise with the yearly seasonality, though only the seasonplots and the STL plots make sense:

seasonplot(ts(hbf$TheValue,frequency=365),pch=19)
plot(stl(ts(hbf$TheValue,frequency=365),s.window="periodic"))

seasonplot_yearly

STL_yearly

Just as for weekly seasonality, no pattern is apparent.

While we are plotting, let's also look at ACF and PACF plots:

acf(hbf$TheValue)
pacf(hbf$TheValue)

ACF

PACF

Yes, a few of the (partial) autocorrelations exceed the significance limits. However, they do so only very slightly, and in a series of more than 1,000 observations, such small exceedances do not really indicate anything relevant. Also, note that there are no obvious periodicities of period 7 in the (P)ACF plots, which we would expect if there were any kind of weekly seasonality.

Bottom line: your data are pretty much not seasonal.

Why do methods like tbats() give you a flat line forecast? Time series (like any other data with a random component) consist of signal and noise. Classical signals are trend, seasonality, autoregression and moving average effects, or the effects of causal drivers (which I assume you do not have). Signal, by definition, is patterns that are forecastable. Everything else is not signal, it is noise. You can call noise "randomness".

Forecasting algorithms try to separate the signal and the noise. They forecast the forecastable part, the signal. It does not make sense to forecast the unforecastable part - it will always make the forecast worse.

There is no seasonal signal in your data, nor is there trend. There is some very little ARMA signal, per the (P)ACF plots. Fitting an ARIMA model picks up on this:

plot(forecast(auto.arima(hbf_ts2),h=100))

arima

Note that the forecast wiggles a tiny little bit at the beginning, but is again essentially a flat line. This is because the ARMA signal is very weak.

Given the pretty much complete lack of signal in your data, I am very sure that such a forecast will be near-optimal. Flat forecasts can beat more "wiggly" ones surprisingly often.

I suggest you try a holdout forecast comparison: hold out the last (say) three months of data, fit a PowerBI model, a TBATS one and an ARIMA one to the remaining data, forecast out into the holdout sample, and compare the Mean Squared Error. I'll bet you a beverage of your choice that PowerBI will not be optimal. (I'll be at a couple of forecasting conferences later this year where you can claim your prize.)

Final question: why does PowerBI give you a very wiggly forecast? I'll be cynical here. I believe that PowerBI does not attempt to give you a good forecast. It wants to give you a forecast that looks sophisticated, so you will believe that PowerBI is sophisticated. Complexity for its own sake will usually make forecasts worse. I very much recommend the special issue of simple vs. complex methods in forecasting in the Journal of Business Research (vol. 68, no. 8, 2015). I do believe that complex methods have their place, but only if there are clear drivers for your series, and there are none such here.

In the end, I'd trust the people behind the forecast package for R much more than the people behind PowerBI. Rob Hyndman, who maintains the package, is probably the world's foremost expert in forecasting. He just stepped down from a decade-long tenure as the editor in chief of the International Journal of Forecasting. He knows what he is doing, and the forecast package is the state of the art, proven over many years. In contrast, after 13 years in forecasting, I have never seen PowerBI in this field. And the screenshot you show does not impress me. It truly looks like snake oil.

$\endgroup$
  • 1
    $\begingroup$ I'm not so sure about this blanket condemnation. I won't defend PowerBI, about which I know nothing, but wish only to point out that (1) the pacf peaks at 7 and 14, although small, support a weekly seasonal model; (2) a robust spread-vs-level plot (based on rolling MADs and medians) indicates a square root transformation could be helpful; and (3) applying the OP's R analysis to the square roots actually produces a varying forecast (from 1.13M to 2.04M). It looks qualitatively like a hybrid of yours and PowerBI's (which does appear to forecast a lot of noise) and shows four quarterly extrema. $\endgroup$ – whuber Jan 23 at 18:21
  • $\begingroup$ Thank you kindly for the detailed analysis - gives me plenty to think about & test! $\endgroup$ – sherifffruitfly Jan 23 at 18:45
  • 2
    $\begingroup$ @whuber: yes, the TBATS model fits a Box-Cox parameter of about 0.5, in line with your suggestion of a square root transform. One may be able to do better than a simple flat forecast. I'll admit that the PowerBI plot raised my hackles. I get this kind of question at my day job, and I'm honestly a bit tired of the entire "if it's more complex, it must be better" idea... $\endgroup$ – Stephan Kolassa Jan 23 at 20:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.