I am looking at time series forecasting using neural networks as described in Hyndman and Athanasopoulos.

They describe Neural Network Auto-Regression models as non-linear generalizations of AR models in the following way:

$NNAR(p,k)$ is a feedforward neural network with one hidden layer where p lagged variables are used as inputs (similar to an $AR(p)$ model, or equivalent to the $p$ in an $ARIMA(p,q,d)$ model) and $k$ is the number of hidden neurons.

$NNAR(p,P,k)_m$ is a feedforward neural network with the same $p$ and $k$, along with $P$ lagged values every m periods to account for seasonality (again, similar to the $p$ and $P$ of a seasonal $ARIMA(p,q,d)(P,Q,D)_m$ model).

No mention is made of the number of output neurons. This is puzzling, since the number of output neurons should be a function of the number of steps ahead that we want to forecast.

In the sunspot example that is given, they forecast 30 years ahead using:

fit <- nnetar(sunspotarea, lambda=0)

It seems to me that they are generating multistep forecasts in one of two ways:

  • Either they are using an output neuron for each future step, so in the sunspot example they gave, they would need 30 output neurons in total. But they would have to specify this before hand (since the topology of the NNet has to specified before training), and that doesn't seem to be the case from the arguments that they feed to the nnetar method.
  • Or they have only one output neuron, and they are feeding the output at each future step $T$ back to the input to calculate the output at $T+1$ (similar to how ARIMA or ETS models calculate forecasts for future time steps) - in which case they are actually using a recurrent neural network (RNN) not a feedforward neural network (contrary to the statement in the text).

Which one of these two approaches are they using?

  • 1
    $\begingroup$ It's a feedforward neural network applied recursively, which is not quite the same thing as a recurrent neural network. $\endgroup$ – Rob Hyndman Apr 19 '18 at 1:15
  • $\begingroup$ It's likely the latter with a single output neuron. It's much easier to lag inputs and then predict. Probably look at Elman-Jordan nets or cascade correlation nets. Cascade correlation nets are best for gradient ascent. $\endgroup$ – user32398 Apr 19 '18 at 2:19
  • $\begingroup$ @RobHyndman thanks for your clarification Professor. So what does that me exactly? That they are feedforward during training and then applied recursively once we are trying to predict new data? Or does it mean something else? $\endgroup$ – Skander H. Apr 19 '18 at 2:48
  • 1
    $\begingroup$ Yes, precisely. I've updated otexts.org/fpp2/nnetar.html to add this explanation. $\endgroup$ – Rob Hyndman Apr 19 '18 at 4:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.