1
$\begingroup$

Given a neural network architecture that has multiple hidden layers and a different number of nodes in each hidden layer (assuming that this is a valid architecture), is it possible to implement feed forward & back prop for such a network using matrix operations? It is clear that matrix operations can be used for feed forward updates when all hidden layers are the same size. But when hidden layers are different sizes, are you forced to iterate over layers to update them with vector operations individually?

$\endgroup$
0
$\begingroup$

is it possible to implement feed forward & back prop for such a network using matrix operations?

yes

when hidden layers are different sizes, are you forced to iterate over layers to update them with vector operations individually?

At each layer of a feed forward neural network you just need one multiplication between the weight matrix of the layer and its input vector (and apply the activation function to each element of the resulting vector). So having a different number of nodes per hidden layer isn't an issue.

$\endgroup$
  • $\begingroup$ I'm trying to do rprop and is it possible to do the derivative portion as a matrix operation too? In python I guess you'd call the derivative funcion with a matrix of values, then add matrices? Too bad I'm not using python. $\endgroup$ – Bing Bang May 14 '18 at 19:52

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.