1
$\begingroup$

I am creating a simple Multi-layered feed forward Neural Network using AForge.net NN library. My NN is a 3 Layered Activation Network trained with Supervised Learning

approach using BackPropogation Learning algorithm.

Following are my initial settings:

//learning rate
learningRate=0.1;

//momentum value
momentum=0;

//alpha value for bipolar sigmoid activation function
sigmoidAlphaValue=2.0;

//number of inputs to network
inputSize=5;

//number of outputs from network
predictionSize=1;

//iterations
iterations=10000;


// create multi-layer neural network
            ActivationNetwork network = new ActivationNetwork(new BipolarSigmoidFunction

(sigmoidAlphaValue), 5, 5 + 1, 3, 1);

//5 inputs
//6 neurons in input layer
//3 neurons in hidden layer
//1 neuron in output layer

// create teacher
BackPropagationLearning teacher = new BackPropagationLearning(network);

// set learning rate and momentum
teacher.LearningRate = learningRate;
teacher.Momentum = momentum;

Now I have some input series which looks like this, 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20

Using window sliding method (as described here) for input as a time series, my input and

expected output array looks something like this

//Iteration #1
double input[][] = new input[0][5] {1,2,3,4,5};
double output[][] = new output[0][0] {6};

//Iteration #2
double input[][] = new input[1][5] {2,3,4,5,6};
double output[][] = new output[1][0] {7};

//Iteration #3
double input[][] = new input[2][5] {3,4,5,6,7};
double output[][] = new output[2][0] {8};
.
.
.
//Iteration #n
double input[][] = new input[n][5] {15,16,17,18,19};
double output[][] = new output[n][0] {20};

After 10k iterations as such using

teacher.RunEpoch(input, output);

my network is successfully trained for the given training set. So now, if I compute using inputs as 4,5,6,7,8 the network successfully gives 9 as answer fantastic!

However, when the input is provided as 21,22,23,24,25 the NN fails to produce 26!

My Question: How do I train my network to accept such unencountered inputs of such fashion to produce a correct sequence pattern as found in training set during learning?

$\endgroup$
  • $\begingroup$ What is the actual output when 26 is expected? $\endgroup$ – Sean Easter Nov 3 '15 at 22:41
  • $\begingroup$ the actual output is 17 $\endgroup$ – dexterslab Nov 9 '15 at 5:50
0
$\begingroup$

I'm going to take a stab at this and say it could be a problem with normalization boundaries.

I'm not familiar with the AForge.net NN library, but at some point your data should be normalized to fit between 0 and 1.

At some point, the normalization process detected 1 as the minimum value and 20 as the max value, and from those bounds, every value is converted to fit between 0 and 1. For example,

1  -> 1/20 = 0.05
...
19 -> 19/20 = 0.95
20 -> 20/20 = 1

When you exceed these bounds later, you're normalization no longer produces values between 0 and 1 and this really wrecks havoc on the network.

25 -> 25/20 = 1.25

What you could do is ensure your normalization factors in your true max and min bounds.

$\endgroup$
  • $\begingroup$ Thanks for the reply. Yes, normalization was the problem here. I switched to Encog framework which provides better normalization techniques. $\endgroup$ – dexterslab Feb 11 '16 at 7:59
  • $\begingroup$ @dexterslab Can you share the type of normalization you used with Encog? I am doing something similar for time series analysis and evaluating both Accord.NET and Encog. Also, did you ever get this solved for Accord.NET? $\endgroup$ – astralmaster Nov 14 '18 at 13:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.