# Formula for number of weights in neural network

I'm trying to find a way to estimate the number of weights in a neural network. Let's look a simple example.

nnetar(1:10) (from the forecast package in R) gives me a 1-1-1 Network with 4 weights.

That makes total sense to me, since we see four arrows in the illustration.

However, nnetar(1:10, xreg=data.frame(10:1,3:12)) gives me a 3-2-1 Network with 11 weights

I don't understand, why the output says that there are 11 weights involved, since I count 12!? Any suggestions?

• The topic of the question seems to be neural networks yet the tool that you're using makes no sense outside time series analysis. Function nnetar is a wrapper of nnet (from package caret) which adapts autoregressive neural networks to time series forecasting. Are you sure this is what you want? Aug 9 '17 at 9:12
• Also, in any type of neural network, the weights are between the input layer and the hidden layers, between hidden layers, and between hidden layers and the output layer. There are no weights outside the input and output layer. In your two figures I'm only seeing 2 and 8 weights, respectively. Aug 9 '17 at 9:16
• Hi Digio, thanks for the reply. I'm using nnetar in a time series context, where it produces good forecasts. So, yes nnetar is want I want. I just gave this example for demonstration purposes.
– shb
Aug 14 '17 at 11:34

The reason you're confused is the fact that function nnetar creates an autoregressive neural network and not a standard neural network. This means that the input layer nodes of the network are:

• the exogenous regressors that you pass with xreg ,
• the autoregressive variables that nnetar creates,
• the bias term.

Running nnetar(1:10, xreg=data.frame(10:1,3:12)), for example, creates by default a NNAR(1,2) model, i.e. a neural network with one lagged term and two hidden nodes.

NNAR(1,2) with two regressors results to a 3-2-1 network where you have:

• 3 nodes in the input layer: $$y_{t-1}$$, $$x_1$$, $$x_2$$
• 2 nodes in the hidden layer
• 1 node in the output layer

If you calculate all weights so far you'll see that you only get 8: $$3 \times 2 + 2 \times 1$$. But then why does nnetar return 11 weights? This is because of the "bias" nodes, which are not really counted in the 3-2-1 network though they are part of it and do carry extra weights. There is one bias node in the input layer and one in the hidden layer which connects only to the output layer. So you have 2 weights from the input layer bias node plus 1 weight from the hidden layer bias node, that makes 3 plus 8 from before, 11 weights in total. You can learn more on this architecture from the documentation of nnetar or Hyndman's new book.

Here's what NNAR(1,2) with 2 regressors looks like:

You can find the number of weights by counting the edges in that network.