I'm trying to build my own neural network class (I know there's 1001 already out there, but this is how I learn best). I'm having a bit of trouble with the sizes of certain vectors. For this I'm working off the following source: Neural Networks.
My problem is calculating the z vector (z is the weighted sum of inputs calculated for each neuron in each layer).
z(2) = w(1) * x + b
w = weight matrix of size (m x n) where m = the number of neuron in the input layer, and n = the number of neuron in second layer x = input matrix of size (s x t) where s is the number of neuron in the input layer and t is the number of training examples. b = input matrix of size (c x d) where c is the number of neurons in the second layer and d is the number of neurons in the input layer
This equation changes for the next layer, but that's not something I'm worried about atm. I know that matrix b is essentially just a vector that's been copied across the width.
Now there's a couple of things I can state a fact from the above: m = s = d, n = c. However when I try and process the above equation none of my matrix sizes match.
Can someone tell me given the list of knowns below what the sizes of the matrix z, w, x and b should be.
1. Input size (number of neurons) 2. Number of neurons in layer 2 3. Number of training examples
I've attempted this sort of thing before and I think I always crash and burn when it comes to initialising the arrays (I'm actually using jagged arrays, but they are rectangular).