3
$\begingroup$

I noticed that in many tutorials with neural networks people difference their time series prior to training/forecasting.

Suppose that we have a window model with many autoregressive terms (say 365 days) that theoretically covers all possible cycles and many samples contain trends, is it really necessary to difference the time series before passing it into MLP training?

As far as I understand, with sufficiently long sample vectors, MLP can understand long seasonalities, as well as trends, and MLP has no assumptions on stationarity.

So, is differencing necessary/advisable/herd mentality when doing time series forecasting using neural networks?

$\endgroup$
3
$\begingroup$

From my understanding, you are correct in assuming you need not difference your input data before running it through your MLP.

One of the most helpful recent papers I have read was this one, written by a Standard Charter Bank researcher. It goes into detail about the process of discovering "what makes a curve move."

A common way to account for this autocorrelation you describe in your data is using Long Short-Term Memory (LSTM) learners. They are a derivative of Recurrent Neural Networks (RNNs) with the ability to "remember" information over a long period of time. There are many helpful resources for understanding these machines, but one I have found most enlightening was this blog post, specifically the "The Core Idea Behind LSTMs" section. In it, it describes the way LSTMs work for time series forecasting with long time horizons like yours. In fact, the keras implementation of LSTMs takes an 3D matrix with dimensions [observations, timesteps (lags), factors] as its input, indicating it is able to handle non-differenced data by understanding the lagged dataset of your input instead.

If you are interested in scaling your data pre-running (highly recommended) I would look at this paper (click "View PDF" in the page), specifically the section "Constructing the Ensemble."

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.