The Back-Propagation Algorithm as described in this guide/introduction is confusing, the final formula generated for del(w)(change in weights) is del(w) formula

del(wji) - Change in weights of synapse from j to i;

eta (n) - learning factor;

Oj - Output of previous layer j;

dj - Desired output from previous layer j;

xi - Input

I have few doubts associated with the formula.

What is the desired output for hidden layer and Input layer??

What is the j and i notation?

What should be the learning factor how can I decide(or generate) the value of learning factor i.e eta(n)?

Note: I want it to work like this (there would be differences in some factors but formulations should be similar..I cannot see any similarity in the formulations).


What is the desired output for hidden layer and input layer?

The hidden layers and input layers don't have a desired output.

Each neuron has an error, which for the output neurons is calculated as target activation - current activation.

Hidden neurons and output calculate their error as follows (psuedocode):

foreach(otherNeuron in myProjectedConnections):
  myError += otherNeuron.error * connection.weight

myProjectedConnections is a list containing all the neurons to which a certain neuron passes on its activation value.

What is the `i` and `j` notation?

Your question contains the answer: del(wji) - Change in weights of synapse from j to i;

A synapse is a fancy word for a connection. So j is a neuron and i is a neuron, with j projecting a connection to i.

What should the learning factor be?

The learning factor should be a factor anywhere in the range of 0-1. If you set your learning factor to 1, a network will learn the output to one test case immediately. If you want to teach it patterns however, choose a value anywhere in the range of 0-0.3, depending on your situation ofcourse.

  • $\begingroup$ Do you know anything about Rprop(Resilient Back propagation), Rprop uses (changes) learning factor to calculate new delw. What if I use Rprop instead of BackProp, which is better? $\endgroup$
    – Chinmaya B
    Apr 29 '17 at 13:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.