Example of time series prediction using neural networks in R

Question

Anyone's got a quick short educational example how to use neural networks (nnet in R for example) for the purpose of prediction?

Here is an example, in R, of a time series

T <- seq(0,20,length=200)
Y <- 1 + 3*cos(4*T+2) +.2*T^2 + rnorm(200)
plot(T,Y,type="l")

This just an example but what I have is jumpy-seasonal data.

Answer 1

Rob Hyndman is doing some active research on forecasting with nueral nets. He recently added the nnetar() function to the forecast package that utilizes the nnet package you reference to fit to time series data.

http://cran.r-project.org/web/packages/forecast/index.html

The example from the help docs:

fit <- nnetar(lynx)
fcast <- forecast(fit)
plot(fcast)

Rob gives more context in this specific section of his online text: Forecasting: principles and practice.

(And a big thanks to Rob obviously.)

Answer 2

Was looking for the same thing and have stumbled upon this question. Since I haven't found an example I've decided to make my own. Be aware, I am no expert in neural nets or forecasting :)

To model time-series effectively with neural nets (nnets) I believe an important property nnets should have is some kind of memory (keeping track of what happened in the past). Therefore, plain feed-forward nets are probably a bad idea. One of the families in nnets that can effectively simulate memory is the family of recurrent neural networks and one of the most known types of recurrent neural networks are probably Elman networks (together with long short-term memory (LSTM) nets I would say). To find more about Elman networks you can go through the original paper introducing the concept or through Wikipedia. In short, they have an additional layer called context used as a type of memory. The following figure (source) illustrates the idea

Luckily for us, there is the RSNNS package in R capable of fitting Elman nets. The package is described in details here.

Now that we have gone through the basics lets see how we can implement an example in R with the RSNNS package.

library(RSNNS)

#
# simulate an arima time series example of the length n
#
set.seed(10001)
n <- 100
ts.sim <- arima.sim(list(order = c(1,1,0), ar = 0.7), n = n-1)

#
# create an input data set for ts.sim
# sw = sliding-window size
#
# the last point of the time series will not be used
#   in the training phase, only in the prediction/validation phase
# 
sw <- 1
X <- lapply(sw:(n-2),
       function(ind){
           ts.sim[(ind-sw+1):ind]
       })
X <- do.call(rbind, X)
Y <- sapply(sw:(n-2),
       function(ind){
           ts.sim[ind+1]
       })

# used to validate prediction properties
# on the last point of the series
newX <- ts.sim[(n-sw):(n-1)]
newY <- ts.sim[n]

# build an elman network based on the input
model <- elman(X, Y,
               size = c(10, 10),
               learnFuncParams = c(0.001),
               maxit = 500,
               linOut = TRUE)

#
# plot the results
#
limits <- range(c(Y, model$fitted.values))

plot(Y, type = "l", col="red",
     ylim=limits, xlim=c(0, length(Y)),
     ylab="", xlab="")
lines(model$fitted.values, col = "green", type="l")

points(length(Y)+1, newY, col="red", pch=16)
points(length(Y)+1, predict(model, newdata=newX),
       pch="X", col="green")

This code should result in the following figure

So what we did with the code is as follows. First, we have created a time-series example (from the ARIMA model). After that we have decoupled/sliced the time-series example into inputs of the form (sw previous points, next point) for all pairs except the last one (with the next point as the last point of the time-series example). The parameter sw is used to define the "sliding window". I won't debate here what is the proper size for the sliding-window but just note that due to Elman networks having memory the sliding-window of size one is more than a reasonable approach (also, take a look at this post).

After the preparations are done we can simply build an Elman network with the elman function. There are two parameters you should be careful about; the size and the learnFuncParams. The size parameter gives you a way to define the size of the network (hidden layer) and the way you choose this parameter is more an art than a science. A rule of thumb for learnFuncParams is to keep it small if it is feasible (your processing power allows you to keep it small/you have enough time to wait :D).

And voila, you have your neural network capable of predicting a/the future point/value. Predictive power of this approach for our example is illustrated in the previous figure. The red curve presents our simulated time series (without the last point) and the green curve what was obtained with the fitted Elman network. The red point denotes the last point (the one that was not used during the fitting process) and the green point what was predicted by the fitted network. Not bad at all :)

This was an example on how to use RNNs (Elman networks) with R to make predictions/forecasting. Some might argue that RNNs are not the best for the problem and that there are better nnet models for forecasting. Since I'm not an expert in the filed I will avoid discussing these issues.

An interesting read if you would like to find more about RNNs is a critical review of RNNs in sequence learning paper.