In order to help myself understand neural networks better, I'm attempting to write the code for a multilayered neural network in Python.
I've written the code for predicting the output, given a set of inputs; however, I'm a bit unsure about how to write the code to do the training.
I know that I need to use the inputs to the matrix in the chain rule to work out the derivative of the weights with respect to the cost of the network.
So at some point, in order to get the value by which to update the weight during training, I will get the connection to which the weight is ascribed, and multiply the input to this connection by an already calculated derivative.
What this means is that for a single set of inputs, I will update the weights using a specific set of values. For another set of inputs, I will need to update the weights with another specific set of values.
Where I am confused is - what do I do when I have multiple sets of inputs?
If I choose to represent the weights of the connections from one layer to another using matrices, does this mean that for each of the multiple inputs, I should do different updates using each of the multiple inputs, meaning that I need to maintain the different sets of weights resulting from each of these different updates? Somehow, this doesn't make sense.
Or do I use all the sets of inputs in my calculations to update the same single set of weights, meaning that I just need to maintain a single set of weights?