1
$\begingroup$

Can the weight initialization algorithm, known as the Nguyen-Widrow Algorithm, be used for a multi-layer perceptron neural network, which consists of more than one hidden layer? (like 6-4-4-3-2)

The paper which introduces this algorithm, only does it for a NN with one hidden layer. How can it be used for multi-hidden layers please?

Improve this question
$\endgroup$
2
  • $\begingroup$ Can I ask why do you need a 6-4-4-3-2 NN? What do you want to classify? $\endgroup$
    – ThiS
    Dec 9, 2012 at 1:14
  • $\begingroup$ That is just an example just to show that I don't mean the normal 1 input 1 hidden and 1 output layered NN $\endgroup$
    – user1724140
    Dec 9, 2012 at 11:22

1 Answer 1

Reset to default
1
$\begingroup$

The Nguyen-Widrow initialization algorithm is the following :

  1. Initialize all weight of hidden layers with (ranged) random values
  2. For each hidden layer
    2.1 calculate beta value, 0.7 * Nth(#neurons of input layer) root of #neurons of current layer
    2.2 for each synapse
    2.1.1 for each weight
    2.1.2 Adjust weight by dividing by norm of weight for neuron and multiplying by beta value

Encog Java Framework

Improve this answer
$\endgroup$
1
  • $\begingroup$ I believe the Encog code is incorrect. The paper specifically discusses H (# of output neurons) and N (# of input neurons) for the given hidden layer in the 2-layer network. The "slicing" is done based on that layers input and number of nodes. It would be illogical to apply information from the input layer to every other layer, however deep. $\endgroup$
    – Greg Kramida
    Dec 17, 2013 at 15:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.