1
$\begingroup$

I am looking for a reasonable estimate of how many total connections (weights) there are in a given neural net. My prior would be that if it is a vanilla ANN with 3 layers of 1200 neurons apiece it would be: (1200^2)*2 . Since each neuron on each layer connects to every other neuron there are 1200^2 between each layer. On a 3 layer ANN there are 2 "in-between" layers. Is this correct? Also, how would this change for RNN's with self-directed loops (and LSTM RNN's)?

$\endgroup$
1
  • $\begingroup$ Do you consider biased weights? $\endgroup$
    – Coder
    Aug 12 '16 at 19:23
1
$\begingroup$

I think you have 1200*2 in-between layers. So, total connection will be (1200^2)*2 (what you wrote is correct.) This is correct, if you are not considering any biased weights. If you add biased weights to each neuron then it would be (1200^2)*2+1200*2 number of connections.

In RNN, each node will be connected twice with itself (Node1 --> Node1 --> Node1). So, (1200^2)*2+(1200^2)*2 (without considering any biased weights). You may generalize it.

$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.