I need some resources to get started on using neural networks for time series forecasting. I am wary of implementing some paper and then finding out that they have greatly over stated the potential of their methods. So if you have experience with the methods you are suggesting it will be even more awesome.
$\begingroup$
$\endgroup$
9
-
$\begingroup$ Please note that NNs are rather... obsolete. $\endgroup$– user88Commented Apr 21, 2011 at 20:09
-
2$\begingroup$ @jason, NN have been characterized as "regression without ethics" as they not only over-fit but make the mistake of "believing the data" rather than "challenging the data for consistency of signal" $\endgroup$– IrishStatCommented Apr 21, 2011 at 21:06
-
13$\begingroup$ NNs are totally not obsolete. They hold the best scores on several important benchmarks currently tackled by the ML community. Also, they are the best multi purpose differentiable function approximator around. Check out the work of Bengio's, Hinton's and lecun's groups of the last 5 years. $\endgroup$– bayerjCommented Apr 21, 2011 at 21:12
-
1$\begingroup$ I read the Bengio & Lecun paper and it does change my mind a bit, but what they are doing is quite different from historical neural nets. The OP wants to work with time series, where there are several methods that can work with univariate time series and actually give you useful information about the time series (DLMs come to mind). If you have data beyond the time series itself, you can use a variety of other methods (LMs, etc) that are straightforward and also illuminating. Why use a black box with unlabeled dials when you can do something understandable? $\endgroup$– WayneCommented Apr 22, 2011 at 13:27
-
8$\begingroup$ Of course the neural nets used back in the 80s/90s are different from what you use today and they are still a very active area of research. Furthermore, you never use neural networks when you care about interpretability. You use them when you care about prediction error. Neural networks are fast and they solve problems other methods fail at. They are nice because they are simple from a conceptual perspective due to the lack of any assumptions about the data you are modeling (except Gaussian noise when used with the squared error). They have their own merits and flaws. $\endgroup$– bayerjCommented Apr 22, 2011 at 22:03
|
Show 4 more comments
2 Answers
$\begingroup$
$\endgroup$
Here's a good quick introduction: intro to neural networks. Note that R has neural-network functionality, so no need to spend any time implementing NN yourself until you've given it a spin and decided it looks promising for your application.
Neural networks are not obsolete, but they have gone through a couple of hype cycles, and then after realizing they don't do everything as was claimed, their reputation goes into a trough for a while (we're currently in one of those). Neural networks are good at certain tasks, and generally are better for tasks in which a human can do a similar task, but cannot explain exactly how they do it.
Neural networks do not give you much insight into the system you're using them to analyze, even after they are trained and operating well. That is, they can predict what will happen (for some systems), but not tell you why. In some cases, that is fine. In others, that is not fine. Generally, if you want or especially if you already have an understanding of the rules of how something works, you can use other techniques.
But, for certain tasks, they work well.
For time-series in particular, see this question's discussion: Proper way of using recurrent neural network for time series analysis
$\begingroup$
$\endgroup$
While it is focussed on statistical pattern recognition, rather than time series forecasting, I would strongly recommend Chris Bishop's book Neural Networks for Pattern Recognition becuase it is the best introduction to neural networks in general, and I think it would be a good idea to get to grips with the potential pitfalls in the use of neural networks in a more simple context, where the problems are more easily visualised an understood. Then move on to the book on recurrent neural networks by Mandic and Chambers. The bishop book is a classic, nobody should use neural nets for anything until they feel confident that they understand the material contained in that book; ANN make it all too easy to shoot yourself in the foot!
I also disagree with mbq, nn are not obsolete, while many problems are better solved with linear models or more modern machine learning techniques (e.g. kernel methods), there are some problems where they work well and other methods don't. It is still a tool that should be in our toolboxes.
answered Apr 22, 2011 at 13:18