I'm implementing an Inception module from scratch, specifically this version with dimensionality reduction, and I'm not sure about how to calculate the input error delta when doing the backpropagation pass.
This is the layer/module structure:
Now, what I'm doing right now is:
- During the forward pass, keep track of the activity/activation for each intermediate step
- Compute the input error delta for each of the 4 "pipelines" individually (using the intermediate activity values when needed from step 1)
- Sum the 4 input error delta tensors (one for each pipeline) to get the global input error delta for the whole module
I'm not sure about the last step though, so my questions are:
Is it right to just sum the 4 input error deltas or should them be averaged together according to some criteria? Any other advice or something I'm doing wrong here?
I couldn't find anything about it on the internet, everyone is just using TensorFlow or some other library that has automatic differentiation, so I'm not sure about what's exactly happing there behind the scenes.
Thank you for your help!