# Implementing Neural Network for time series

I am currently working on neural networks for time series forecasting. My doubt is: do we need to take into account issues like trend, non-stationarity and seasonality while using neural networks instead of the box jenkins methodology?

If yes, how do we do that? For instance, do we need to change the cost function or other parameters? I am currently using nnet and neuralnet packages in R but I do not find anything about these issues in the related documentation.

If not, then does it imply that the model learns the seasonality from data itself? For instance, suppose there is a time series with an upward trend, along with some variations. Could I assume that if we fit a neural network on such series then it might learn just an interpolation?

• You should look at the R forecast package, specifically method nnetar by Rob Hyndman for other insight into how people have dealt with this. – learner Jan 1 '14 at 13:49
• I would suggest to give a look to the Hyndman's free book about Forecasting here: otexts.org/fpp – Matteo De Felice Jan 3 '14 at 19:38
• @learner and matteo Thanks for your suggestions . I already checked them out . My concerns are regarding a multi-variate time series. The issues of stationarity and seasonality can be dealt in multivariate time series when we are using VAR (Vector Autoregression) models. I was looking for some parallel analogy in Neural Networks also. Another surprising fact, There are not many research papers published dealing with multivariate time seires using neural network, Most of them are of 1990s . Does it suggests that Neural Network for Time Series is not a good approach – NG_21 Jan 5 '14 at 4:19
• one recent paper related to the problem arxiv.org/pdf/1703.04122.pdf – mic Mar 15 '17 at 10:10
• Centering, scaling, and differentiation are your friends when it comes to dealing with non-stationarity. Centering and scaling can be performed by utilizing rolling center and scale metrics (rolling means/medians and standard deviations / median absolute deviations). If you have a strong a priori sense that your response data exhibit seasonality, then quantifying that seasonality and including it as a predictor is also a good idea. But without that there's no magic by which the model will identify said seasonality. Remember that feedforward neural nets assume all observations to be i.i.d. – Josh Feb 6 '19 at 20:08

## 2 Answers

There is an editorial by Chris Chatfield in the International Journal of Forecasting 9 (1993) 1-3 entitled "Neural networks: Forecasting breakthrough or passing fad?" It is focused on comparing ARIMA models and neural networks. He warns that sometimes apparently successful applications of neural networks in business and economic forecasting are reported without comparing the results with any more established alternatives. Chatfield concludes:

In summary it is possible that neural nets will outperform standard forecasting
procedures when a fair comparison is made, at least for certain types of situation, but
there is little systematic evidence of this as yet.


There is also a paper co-authored by Chatfield published in Applied Statistics. It compares NN with Box-Jenkins and Holt-Winters and reports on many potential problems in using NN for forecasting. The authors advise "it is unwise to apply NN blindly in a 'black-box' as has sometimes been suggested", which I think answers your questions.

If you are currently working on neural networks for time series forecasting, I would suggest that you build your own collection of quality references. The two references above is a good start. Brian Ripley discussed NN for time series neither in this piece of R news nor in his "Pattern Recognition and Neural Network" book, but you probably can find his work on neural networks for time series prediction elsewhere.

You may also check this review paper: Forecasting with artificial neural networks: The state of the art in the IJF, even though it is dated 1998. It has references to work on applying NN to multivariate time series problem. In particular, it says that

Gorr (1994) believes that ANNs can be more appropriate for the following situations:
(1) large data sets;
(2) problems with nonlinear structure;
(3) the multivariate time series forecasting problems.


There is also "...A Review from a Statistical Perspective" (and Leo Breimann remarked in his commentary to it that "room is left for other statistical perspective") published in Statistical Science, Vol. 9, No. 1 (Feb., 1994)

• Thanks, It helped. But my major concerns are regarding multivariate time series and Neural Network. I havent seen much research in this field. Most of the papers are from 1990s Is it so that Neural Networks are giving way to other machine learning techniques like SVR, KNN etc in the field of time series? – NG_21 Jan 5 '14 at 14:55
• I expanded my answer to include a link to a review paper discussing (among other things) multivariate time series. As to the current trends in the use of NN and related techniques, try and find a more recent review paper. – Hibernating Jan 5 '14 at 15:34

Some Googling for specifically neural networks and seasonality leads to this paper, Neural network forecasting for seasonal and trend time series, Zhang and Qi, European Journal of Operational Research, V.160, 2, 16 January 2005, 501–514. In this paper the authors sought to compare the Box-Jenkins approach with a neural network approach. From the abstract:

We find that neural networks are not able to capture seasonal or trend variations effectively with the unpreprocessed raw data and either detrending or deseasonalization can dramatically reduce forecasting errors. Moreover, a combined detrending and deseasonalization is found to be the most effective data preprocessing approach.

They conclude that accounting for trend and seasonality in your preprocessing steps is a good idea.

• Thanks for actively responding. I had already checked this paper. As far as univariate time series are concerned, We already have some good references. What we don't have much quality is regarding neural network implementation for " Multivariate Time Series" and its comparision with VAR (Vector Auto Regression) models – NG_21 Jan 5 '14 at 14:50
• Ah, I am very sorry then, but your estimation that research has dropped off is most likely correct. However, as I'm sure it seems to you, it seems likely to me that the multivariate case would at least be as impacted by seasonality/trend as the univariate, if not more so. – learner Jan 5 '14 at 20:34