I've heard a bit about using neural networks to forecast time series.
How can I compare, which method for forecasting my time-series (daily retail data) is better: auto.arima(x), ets(x) or nnetar(x).
I can compare auto.arima with ets by AIC or BIC. But how I can compare them with neural networks?
For example:
> dput(x)
c(1774, 1706, 1288, 1276, 2350, 1821, 1712, 1654, 1680, 1451,
1275, 2140, 1747, 1749, 1770, 1797, 1485, 1299, 2330, 1822, 1627,
1847, 1797, 1452, 1328, 2363, 1998, 1864, 2088, 2084, 594, 884,
1968, 1858, 1640, 1823, 1938, 1490, 1312, 2312, 1937, 1617, 1643,
1468, 1381, 1276, 2228, 1756, 1465, 1716, 1601, 1340, 1192, 2231,
1768, 1623, 1444, 1575, 1375, 1267, 2475, 1630, 1505, 1810, 1601,
1123, 1324, 2245, 1844, 1613, 1710, 1546, 1290, 1366, 2427, 1783,
1588, 1505, 1398, 1226, 1321, 2299, 1047, 1735, 1633, 1508, 1323,
1317, 2323, 1826, 1615, 1750, 1572, 1273, 1365, 2373, 2074, 1809,
1889, 1521, 1314, 1512, 2462, 1836, 1750, 1808, 1585, 1387, 1428,
2176, 1732, 1752, 1665, 1425, 1028, 1194, 2159, 1840, 1684, 1711,
1653, 1360, 1422, 2328, 1798, 1723, 1827, 1499, 1289, 1476, 2219,
1824, 1606, 1627, 1459, 1324, 1354, 2150, 1728, 1743, 1697, 1511,
1285, 1426, 2076, 1792, 1519, 1478, 1191, 1122, 1241, 2105, 1818,
1599, 1663, 1319, 1219, 1452, 2091, 1771, 1710, 2000, 1518, 1479,
1586, 1848, 2113, 1648, 1542, 1220, 1299, 1452, 2290, 1944, 1701,
1709, 1462, 1312, 1365, 2326, 1971, 1709, 1700, 1687, 1493, 1523,
2382, 1938, 1658, 1713, 1525, 1413, 1363, 2349, 1923, 1726, 1862,
1686, 1534, 1280, 2233, 1733, 1520, 1537, 1569, 1367, 1129, 2024,
1645, 1510, 1469, 1533, 1281, 1212, 2099, 1769, 1684, 1842, 1654,
1369, 1353, 2415, 1948, 1841, 1928, 1790, 1547, 1465, 2260, 1895,
1700, 1838, 1614, 1528, 1268, 2192, 1705, 1494, 1697, 1588, 1324,
1193, 2049, 1672, 1801, 1487, 1319, 1289, 1302, 2316, 1945, 1771,
2027, 2053, 1639, 1372, 2198, 1692, 1546, 1809, 1787, 1360, 1182,
2157, 1690, 1494, 1731, 1633, 1299, 1291, 2164, 1667, 1535, 1822,
1813, 1510, 1396, 2308, 2110, 2128, 2316, 2249, 1789, 1886, 2463,
2257, 2212, 2608, 2284, 2034, 1996, 2686, 2459, 2340, 2383, 2507,
2304, 2740, 1869, 654, 1068, 1720, 1904, 1666, 1877, 2100, 504,
1482, 1686, 1707, 1306, 1417, 2135, 1787, 1675, 1934, 1931, 1456)
Using auto.arima:
y=auto.arima(x)
plot(forecast(y,h=30))
points(1:length(x),fitted(y),type="l",col="green")
> summary(y)
Series: x
ARIMA(5,1,5)
Coefficients:
ar1 ar2 ar3 ar4 ar5 ma1 ma2 ma3 ma4 ma5
0.2560 -1.0056 0.0716 -0.5516 -0.4822 -0.9584 1.2627 -1.0745 0.8545 -0.2819
s.e. 0.1014 0.0778 0.1296 0.0859 0.0844 0.1184 0.1322 0.1289 0.1388 0.0903
sigma^2 estimated as 58026: log likelihood=-2191.97
AIC=4405.95 AICc=4406.81 BIC=4447.3
Training set error measures:
ME RMSE MAE MPE MAPE MASE
Training set 1.457729 240.5059 173.9242 -2.312207 11.62531 0.6157512
Using ets:
fit <- ets(x)
plot(forecast(fit,h=30))
points(1:length(x),fitted(fit),type="l",col="red")
> summary(fit)
ETS(M,N,N)
Call:
ets(y = x)
Smoothing parameters:
alpha = 0.0449
Initial states:
l = 1689.128
sigma: 0.2094
AIC AICc BIC
5570.373 5570.411 5577.897
Training set error measures:
ME RMSE MAE MPE MAPE MASE
Training set 7.842061 359.3611 276.4327 -4.81967 17.98136 0.9786665
In this case auto.arima fits better then ets.
Let's try sing neural network:
library(caret)
fit <- nnetar(x)
plot(forecast(fit,h=60))
points(1:length(x),fitted(fit),type="l",col="green")
From the graph, I can see, that neural network model fits quite well, but how can I compare it with auto.arima/ets? How can I compute AIC?
Another question is, how to add confidence interval for neural network,if it is possible, like it is added automatically for auto.arima/ets.?
Any help and advises would be really appreciated.