Implementing Neural Network for time series

Question

I am currently working on neural networks for time series forecasting. My doubt is: do we need to take into account issues like trend, non-stationarity and seasonality while using neural networks instead of the box jenkins methodology?

If yes, how do we do that? For instance, do we need to change the cost function or other parameters? I am currently using nnet and neuralnet packages in R but I do not find anything about these issues in the related documentation.

If not, then does it imply that the model learns the seasonality from data itself? For instance, suppose there is a time series with an upward trend, along with some variations. Could I assume that if we fit a neural network on such series then it might learn just an interpolation?

Answer 1

Some Googling for specifically neural networks and seasonality leads to this paper, Neural network forecasting for seasonal and trend time series, Zhang and Qi, European Journal of Operational Research, V.160, 2, 16 January 2005, 501–514. In this paper the authors sought to compare the Box-Jenkins approach with a neural network approach. From the abstract:

We find that neural networks are not able to capture seasonal or trend variations effectively with the unpreprocessed raw data and either detrending or deseasonalization can dramatically reduce forecasting errors. Moreover, a combined detrending and deseasonalization is found to be the most effective data preprocessing approach.

They conclude that accounting for trend and seasonality in your preprocessing steps is a good idea.

Answer 2

There is an editorial by Chris Chatfield in the International Journal of Forecasting 9 (1993) 1-3 entitled "Neural networks: Forecasting breakthrough or passing fad?" It is focused on comparing ARIMA models and neural networks. He warns that sometimes apparently successful applications of neural networks in business and economic forecasting are reported without comparing the results with any more established alternatives. Chatfield concludes:

In summary it is possible that neural nets will outperform standard forecasting 
procedures when a fair comparison is made, at least for certain types of situation, but
there is little systematic evidence of this as yet.

There is also a paper co-authored by Chatfield published in Applied Statistics. It compares NN with Box-Jenkins and Holt-Winters and reports on many potential problems in using NN for forecasting. The authors advise "it is unwise to apply NN blindly in a 'black-box' as has sometimes been suggested", which I think answers your questions.

If you are currently working on neural networks for time series forecasting, I would suggest that you build your own collection of quality references. The two references above is a good start. Brian Ripley discussed NN for time series neither in this piece of R news nor in his "Pattern Recognition and Neural Network" book, but you probably can find his work on neural networks for time series prediction elsewhere.

You may also check this review paper: Forecasting with artificial neural networks: The state of the art in the IJF, even though it is dated 1998. It has references to work on applying NN to multivariate time series problem. In particular, it says that

Gorr (1994) believes that ANNs can be more appropriate for the following situations:   
(1) large data sets; 
(2) problems with nonlinear structure; 
(3) the multivariate time series forecasting problems.

There is also "...A Review from a Statistical Perspective" (and Leo Breimann remarked in his commentary to it that "room is left for other statistical perspective") published in Statistical Science, Vol. 9, No. 1 (Feb., 1994)