How can I calculate the number of parameters in an artificial neural network in order to calculate its AIC?

  • $\begingroup$ This question seems perfectly clear to me. $\endgroup$ – gung - Reinstate Monica Sep 28 '15 at 17:16
  • $\begingroup$ You can use the command classifier.summary() from sklear class. $\endgroup$ – Shekhar Shinde Feb 23 '18 at 0:09

Every connection that is learned in a feedforward network is a parameter. Here is an image of a generic network from Wikipedia:

enter image description here

This network is fully connected, although networks don't have to be (e.g., designing a network with receptive fields improves edge detection in images). With a fully connected ANN, the number of connections is simply the sum of the product of the numbers of nodes in connected layers. In the image above, that is $(3\times 4) + (4\times 2) = 20$. That image does not show any bias nodes, but many ANNs do have them; if so, include the bias node in the total for that layer. More generally (e.g., if your ANN isn't fully connected), you can simply count the connections.

  • $\begingroup$ Connections can be non-unique (see ieeexplore.ieee.org/document/714176). Hence, is it okay to simply count the connections? Maybe we should distinguish between parameter and hyperparameter? $\endgroup$ – Julian Jul 24 at 13:50

Neural network is just a function of functions of functions ... (as dictated by the architecture of the model). If the resulting function can't be simplified then the total number of parameters (sum of all number of parameters from each nodes) in the model is the number you want for the AIC calculation.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.