I am looking at time series forecasting using neural networks as described in Hyndman and Athanasopoulos.
They describe Neural Network Auto-Regression models as non-linear generalizations of AR models in the following way:
$NNAR(p,k)$ is a feedforward neural network with one hidden layer where p lagged variables are used as inputs (similar to an $AR(p)$ model, or equivalent to the $p$ in an $ARIMA(p,q,d)$ model) and $k$ is the number of hidden neurons.
$NNAR(p,P,k)_m$ is a feedforward neural network with the same $p$ and $k$, along with $P$ lagged values every m periods to account for seasonality (again, similar to the $p$ and $P$ of a seasonal $ARIMA(p,q,d)(P,Q,D)_m$ model).
No mention is made of the number of output neurons. This is puzzling, since the number of output neurons should be a function of the number of steps ahead that we want to forecast.
In the sunspot example that is given, they forecast 30 years ahead using:
fit <- nnetar(sunspotarea, lambda=0) autoplot(forecast(fit,h=30))
It seems to me that they are generating multistep forecasts in one of two ways:
- Either they are using an output neuron for each future step, so in the sunspot example they gave, they would need 30 output neurons in total. But they would have to specify this before hand (since the topology of the NNet has to specified before training), and that doesn't seem to be the case from the arguments that they feed to the nnetar method.
- Or they have only one output neuron, and they are feeding the output at each future step $T$ back to the input to calculate the output at $T+1$ (similar to how ARIMA or ETS models calculate forecasts for future time steps) - in which case they are actually using a recurrent neural network (RNN) not a feedforward neural network (contrary to the statement in the text).
Which one of these two approaches are they using?