I am creating a simple Multi-layered feed forward Neural Network using AForge.net NN library. My NN is a 3 Layered Activation Network trained with Supervised Learning
approach using BackPropogation Learning algorithm.
Following are my initial settings:
//learning rate
learningRate=0.1;
//momentum value
momentum=0;
//alpha value for bipolar sigmoid activation function
sigmoidAlphaValue=2.0;
//number of inputs to network
inputSize=5;
//number of outputs from network
predictionSize=1;
//iterations
iterations=10000;
// create multi-layer neural network
ActivationNetwork network = new ActivationNetwork(new BipolarSigmoidFunction
(sigmoidAlphaValue), 5, 5 + 1, 3, 1);
//5 inputs
//6 neurons in input layer
//3 neurons in hidden layer
//1 neuron in output layer
// create teacher
BackPropagationLearning teacher = new BackPropagationLearning(network);
// set learning rate and momentum
teacher.LearningRate = learningRate;
teacher.Momentum = momentum;
Now I have some input series which looks like this, 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20
Using window sliding method (as described here) for input as a time series, my input and
expected output array looks something like this
//Iteration #1
double input[][] = new input[0][5] {1,2,3,4,5};
double output[][] = new output[0][0] {6};
//Iteration #2
double input[][] = new input[1][5] {2,3,4,5,6};
double output[][] = new output[1][0] {7};
//Iteration #3
double input[][] = new input[2][5] {3,4,5,6,7};
double output[][] = new output[2][0] {8};
.
.
.
//Iteration #n
double input[][] = new input[n][5] {15,16,17,18,19};
double output[][] = new output[n][0] {20};
After 10k iterations as such using
teacher.RunEpoch(input, output);
my network is successfully trained for the given training set. So now, if I compute using inputs as 4,5,6,7,8 the network successfully gives 9 as answer fantastic!
However, when the input is provided as 21,22,23,24,25 the NN fails to produce 26!
My Question: How do I train my network to accept such unencountered inputs of such fashion to produce a correct sequence pattern as found in training set during learning?