# Simple Neural Network for time series prediction

I am creating a simple Multi-layered feed forward Neural Network using AForge.net NN library. My NN is a 3 Layered Activation Network trained with Supervised Learning

approach using BackPropogation Learning algorithm.

Following are my initial settings:

//learning rate
learningRate=0.1;

//momentum value
momentum=0;

//alpha value for bipolar sigmoid activation function
sigmoidAlphaValue=2.0;

//number of inputs to network
inputSize=5;

//number of outputs from network
predictionSize=1;

//iterations
iterations=10000;

// create multi-layer neural network
ActivationNetwork network = new ActivationNetwork(new BipolarSigmoidFunction

(sigmoidAlphaValue), 5, 5 + 1, 3, 1);

//5 inputs
//6 neurons in input layer
//3 neurons in hidden layer
//1 neuron in output layer

// create teacher
BackPropagationLearning teacher = new BackPropagationLearning(network);

// set learning rate and momentum
teacher.LearningRate = learningRate;
teacher.Momentum = momentum;


Now I have some input series which looks like this, 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20

Using window sliding method (as described here) for input as a time series, my input and

expected output array looks something like this

//Iteration #1
double input[][] = new input {1,2,3,4,5};
double output[][] = new output {6};

//Iteration #2
double input[][] = new input {2,3,4,5,6};
double output[][] = new output {7};

//Iteration #3
double input[][] = new input {3,4,5,6,7};
double output[][] = new output {8};
.
.
.
//Iteration #n
double input[][] = new input[n] {15,16,17,18,19};
double output[][] = new output[n] {20};


After 10k iterations as such using

teacher.RunEpoch(input, output);


my network is successfully trained for the given training set. So now, if I compute using inputs as 4,5,6,7,8 the network successfully gives 9 as answer fantastic!

However, when the input is provided as 21,22,23,24,25 the NN fails to produce 26!

My Question: How do I train my network to accept such unencountered inputs of such fashion to produce a correct sequence pattern as found in training set during learning?

• What is the actual output when 26 is expected? – Sean Easter Nov 3 '15 at 22:41
• the actual output is 17 – dexterslab Nov 9 '15 at 5:50

I'm going to take a stab at this and say it could be a problem with normalization boundaries.

I'm not familiar with the AForge.net NN library, but at some point your data should be normalized to fit between 0 and 1.

At some point, the normalization process detected 1 as the minimum value and 20 as the max value, and from those bounds, every value is converted to fit between 0 and 1. For example,

1  -> 1/20 = 0.05
...
19 -> 19/20 = 0.95
20 -> 20/20 = 1


When you exceed these bounds later, you're normalization no longer produces values between 0 and 1 and this really wrecks havoc on the network.

25 -> 25/20 = 1.25


What you could do is ensure your normalization factors in your true max and min bounds.

• Thanks for the reply. Yes, normalization was the problem here. I switched to Encog framework which provides better normalization techniques. – dexterslab Feb 11 '16 at 7:59
• @dexterslab Can you share the type of normalization you used with Encog? I am doing something similar for time series analysis and evaluating both Accord.NET and Encog. Also, did you ever get this solved for Accord.NET? – astralmaster Nov 14 '18 at 13:50