$\partial E/\partial b^3$ should equal a matrix of dimensions: $3 \times 1$, in order to make the subtraction possible by the former $b^3$. The batch size of 5, however, made the dimensions of the $b^3$ matrix equal to $3 \times 5$, which is problematic as you now can't subtract $\partial E/\partial b^3$ because of the dimension mismatch.
What went wrong? Am I supposed to take the average of each row of the $\partial E/\partial b^3$ matrix or perhaps just accumulate each row? Or is it something completely different?
My reflection At the moment I am thinking that what I have done up to now is correct, however I just need to either accumulate or take the average of all the numbers in each row in the matrix $∂E/∂b3$. This way the matrix will be of size $3\times1$ as wanted, and it also makes intuitive sense in the way that I am updating the bias based on an error calculated over an entire batch size, therefor either accumulating or taking the average would make sense. However as I mentioned I am not sure which one it is, or if it is even the right choice.
Any help is highly appreciated!