1
$\begingroup$

Can the weight initialization algorithm, known as the Nguyen-Widrow Algorithm, be used for a multi-layer perceptron neural network, which consists of more than one hidden layer? (like 6-4-4-3-2)

The paper which introduces this algorithm, only does it for a NN with one hidden layer. How can it be used for multi-hidden layers please?

Improve this question
$\endgroup$
2
  • $\begingroup$ Can I ask why do you need a 6-4-4-3-2 NN? What do you want to classify? $\endgroup$ – ThiS Dec 9 '12 at 1:14
  • $\begingroup$ That is just an example just to show that I don't mean the normal 1 input 1 hidden and 1 output layered NN $\endgroup$ – user1724140 Dec 9 '12 at 11:22
1
$\begingroup$

The Nguyen-Widrow initialization algorithm is the following :

  1. Initialize all weight of hidden layers with (ranged) random values
  2. For each hidden layer
    2.1 calculate beta value, 0.7 * Nth(#neurons of input layer) root of #neurons of current layer
    2.2 for each synapse
    2.1.1 for each weight
    2.1.2 Adjust weight by dividing by norm of weight for neuron and multiplying by beta value

Encog Java Framework

Improve this answer
$\endgroup$
1
  • $\begingroup$ I believe the Encog code is incorrect. The paper specifically discusses H (# of output neurons) and N (# of input neurons) for the given hidden layer in the 2-layer network. The "slicing" is done based on that layers input and number of nodes. It would be illogical to apply information from the input layer to every other layer, however deep. $\endgroup$ – Greg Kramida Dec 17 '13 at 15:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.