1
$\begingroup$

I was wondering whether an artificial neural network regression, like ARIMA, requires statistically insignificant residual autocorrelation -- and, if so, why?

I presume that, if I am using the ANN model to forecast prediction intervals, statistically insignificant heteroscedasticity and statistically significant residual normality are requisites. Is this correct?

$\endgroup$
7
  • 1
    $\begingroup$ I do not think ARIMA is an artificial neural network regression. It looks and works pretty differently. $\endgroup$ Jun 15, 2020 at 16:33
  • $\begingroup$ Right. ARIMA is a time series model. I guess my question is: do all time series model require insignificant serial autocorrelation -- or is that specific to linear models? As for the normality / heteroskedasticity question, I presume those assumptions hold as long as I am building my intervals using Upper / Lower Bound = y_hat +/- z * SD. Is that correct? $\endgroup$
    – CasusBelli
    Jun 15, 2020 at 16:37
  • 1
    $\begingroup$ I think a brief answer is a Yes. $\endgroup$ Jun 15, 2020 at 16:38
  • $\begingroup$ Thank you my friend. Also, not to overwhelm with questions, am I essentially confined to White's heteroskedasticity test if my residuals are not normally distributed? I believe the BP test requires residual normality ... Otherwise, for building prediction intervals,I think I can circumvent the normality requirements for the residuals by fitting them to another distribution, estimating the parameters, generating random numbers using those, and then selecting the values corresponding to the desired quantiles. Sorry to bombard with questions -- and thank you for your input! :) $\endgroup$
    – CasusBelli
    Jun 15, 2020 at 17:45
  • 1
    $\begingroup$ Yes, using quantiles of the empirical distribution of the residuals for contructing a prediction interval could be OK. Regarding White test vs. BP test, I simply don't remember. I would have to look this up somewhere, but you could do the same equally well. $\endgroup$ Jun 15, 2020 at 19:36

1 Answer 1

1
$\begingroup$

Unlike ARMA and ARIMA, there are no assumptions required whatsoever when performing time series modeling with Neural Networks. This is because Neural Networks are universal approximators, and with a deep and wide enough architecture, they can approximate any arbitrary function. Moreover, they are not based on any statistical assumptions, and are instead "brute force/mechanical" non parametric function approximators, so considerations like heteroscedasticity, non-correlated errors, stationarity, etc...don't come into play the way they would for a regression model or for ARIMA.

That being said, in practice, there is some evidence that performing some of the customary transformations, like stabilizing the variance with a power transform, detrending and/or deseasonalizing, improves the performance of various deep learning models on forecasting tasks. In my personal experience, just performing normalization (e.g. using the sklearn standardscaler or minmaxscaler) gives results that are just as good as a more complex preprocessing like a power transform, at least in the LSTM case. The time series in my use case were all short and had little or no trend at all to begin with, only seasonal patterns.

But opinions and findings differ: Rob Hyndman's NNETAR model uses a power transform. Smyl, in the LSTM model that won the M4 competition, removed both the seasonality and the trend. Hansika Hewamalage, Christoph Bergmeir , Kasun Bandara say that it depends: Sometimes detrending is enough, sometime removing the seasonality is necessary as well.

To reiterate, Neural Networks are universal function approximators that do not rely on any formally defined statistical/stochastic process, and hence none of the requirements of ARIMA are relevant in theory, but in practice, trying to enforce some of these requirements might give better results than just feeding the time series directly to the network.

You mention:

I presume that, if I am using the ANN model to forecast prediction intervals, statistically insignificant heteroscedasticity and statistically significant residual normality are requisites. Is this correct?

No. As I said above, not at all. Moreover, one neural network forecasting model at least goes completely in the other direction and does full non-parametric density forecasts, that are based on the idea that we can't make any assumptions whatsoever on the statistical properties of our time series.


In response to @Richard Harris' comment on stationarity being necessary, the below forecast for the Air Passengers time series was generated using an LSTM model, where the only preprocessing was normalizing via the variance and the mean of the data set ($x^* = \frac{x-\mu_x}{\sigma_x})$. Here is the code - using Tensorflow. Note that this won't run on later versions (TF 2.x) and would require you to downgrade to TF 1.12 or 1.13.

enter image description here

$\endgroup$
7
  • $\begingroup$ Regarding your first paragraph (and related ideas later on): without some form of stationarity, no method will be able to generalize from a sample to the data generating process and new samples from it. Neural networks are no exception. $\endgroup$ Jun 17, 2020 at 9:02
  • $\begingroup$ @RichardHardy I disagree. I've tried myself. $\endgroup$
    – Skander H.
    Jun 17, 2020 at 9:05
  • $\begingroup$ @RichardHardy see plot and link to code I posted. $\endgroup$
    – Skander H.
    Jun 17, 2020 at 9:27
  • $\begingroup$ Clearly, the Air Passengers data is stationary in some sense. It is trend+seasonality stationary, I guess. Try a couple of independent random walks and see how successfully a neural network predicts (out of sample) one given the other. It will fail miserably, just as any other method would. $\endgroup$ Jun 17, 2020 at 9:42
  • $\begingroup$ @RichardHardy Interesting thought. Can you clarify for me though: To the best of my knowledge no time series model can predict a random walk, other than a random walk? $\endgroup$
    – Skander H.
    Jun 17, 2020 at 9:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.