I have a problem in my neural network relating to the bias vector.
I'm using this source as a reference.
Now, I'm aware that the bias vector is essentially a n x 1 vector (where n is the number of neuron in the layer) but is treated as a n x m vector (where m is the number of training examples) by copying the values of the n x 1 into m rows. But when I calculate my delta(l)'s this gives me a n x m matrix that has different values in each location of the matrix. How can I then turn my bias matrix back into a n x 1 vector?
Example: A network with 1 inputs neuron, 2 hidden layers (with 3 neurons in each) and 1 output neuron will have a delta(2) that is a 3 x m matrix (where m is the number of training examples).
When since gradB(1) is equal to delta(1+1), gradB(1) will also be of size 3 x m. Each value in the matrix is different from the others so how do I update my bias vector b(1) (dimensions 3 x 1) from the 3 x m matrix? (ignoring all the scalar production relating to 1/m and alpha).