7
$\begingroup$

After working on Backpropagation Neural Network and ARIMA Time Series Model, I asked myself which one is better, but can't figure out the answer. They both use different approaches on the same problem (future prediction). Please can someone help me stating the obvious.

Backpropagation(C++):

 typedef struct {                     /* A LAYER OF A NET:                     */
    INT           Units;         /* - number of units in this layer       */
    REAL*         Output;        /* - output of ith unit                  */
    REAL*         Error;         /* - error term of ith unit              */
    REAL**        Weight;        /* - connection weights to ith unit      */
    REAL**        WeightSave;    /* - saved weights for stopped training  */
    REAL**        dWeight;       /* - last weight deltas for momentum     */
} LAYER;

typedef struct {                     /* A NET:                                */
    LAYER**       Layer;         /* - layers of this net                  */
    LAYER*        InputLayer;    /* - input layer                         */
    LAYER*        OutputLayer;   /* - output layer                        */
    REAL          Alpha;         /* - momentum factor                     */
    REAL          Eta;           /* - learning rate                       */
    REAL          Gain;          /* - gain of sigmoid function            */
    REAL          Error;         /* - total net error                     */
} NET;

ARIMA(R):

arima(stockadj,order=c(best.model[1],best.model[2],best.model[3]),xreg=1:n)
stockfor<-predict(stockari,h=100,newxreg=(n+1):(n+100))
ts.plot(stockadj,stockfor$pred,ylab="Original+Predicted Values",main="Forecast")   
$\endgroup$

2 Answers 2

5
$\begingroup$

It looks like you are using both of these models for time-series forecasts. I would cross-validate both models and compare their out-of-sample error.

$\endgroup$
5
  • 1
    $\begingroup$ How are you handling lagged effects in your neural network? $\endgroup$
    – AdamO
    Commented Apr 11, 2013 at 21:37
  • $\begingroup$ @AdamO I have no idea-- ask AbhilashK $\endgroup$
    – Zach
    Commented Apr 11, 2013 at 23:19
  • $\begingroup$ If I were to check their out-of-sample error, it would only give me the errors w.r.t the method. What I want is theoretical comparison between the 2 methods. As for my neural network,it just calculates the forecast nothing else. $\endgroup$ Commented Apr 12, 2013 at 3:10
  • 2
    $\begingroup$ @AbhilashK if you choose to ignore empirical evidence, "better" becomes fairly subjective... $\endgroup$
    – Zach
    Commented Apr 12, 2013 at 4:22
  • 1
    $\begingroup$ As far as I can see, the OP is asking which is better in general, which is almost always impossible to say. As far as I can tell, "better" boils down to how well your data fits the assumptions of the procedure. (If the data fits the assumptions of two procedures "equally", then the procedure that makes stronger assumptions would tend to be better.) $\endgroup$
    – Wayne
    Commented Apr 13, 2013 at 14:33
1
$\begingroup$

NN ignore outliers. If you ignore outliers, then you are in big trouble.

Your ARIMA model is also ignoring outliers so then you are also in big trouble.

As for cross-validating, that is for those that are fitting models to data instead of actually modeling. Only the 849 page text book "Principles of Forecasting" agrees with me on this statement, BUT if you have taken care of all the things you need to then you can be so dumb. See the reference here.

4.6 Obtain the most recent data

See more on outliers here.

$\endgroup$
5
  • $\begingroup$ can you say a little more about the way in which Neural Networks ignore outliers? $\endgroup$ Commented Apr 13, 2013 at 8:29
  • $\begingroup$ I will point to some other posts and they make no discussion of outliers. Perhaps NN have advanced with respect to outliers, but I am not aware.... if you take the time series 1,9,1,9,1,9,1,9,1,5...will it find the 5 to be an outlier? stats.stackexchange.com/questions/8000/… and stats.stackexchange.com/questions/9842/… and stats.stackexchange.com/questions/10162/… $\endgroup$
    – Tom Reilly
    Commented Apr 15, 2013 at 13:47
  • $\begingroup$ Thank you, I understand your comments a little better now. By "ignore" you mean that NN give outliers no special significance. In general I agree, though I imagine you could add a weighting term to the instances that you felt might be suspect. I think I have more of a philosophical challenge when it comes to identifying outliers, without extra information I'm not sure there is any reason to suspect 5 to be an outlier in the series, unless you make assumptions not supported by the data from an empirical standpoint. $\endgroup$ Commented Apr 15, 2013 at 15:46
  • $\begingroup$ In time series analysis, you need to make an effort to identify them so ignoring them doesn't work. I will point you to the work of Box, Tiao, Tsay and Fox for more on the importance. The 1,9 pattern is clear as a bell. The 5 is an outlier or "inlier" and is a great example. I have more.... $\endgroup$
    – Tom Reilly
    Commented Apr 15, 2013 at 20:21
  • $\begingroup$ in fact, if the loss function used for training is the mean squared error, the network actually gives more importance to outliers since large errors become even larger when they're squared $\endgroup$
    – user91213
    Commented Dec 15, 2015 at 0:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.