I'm trying to study a basic model like ResNet and how many operations it does and memory usage during backward-pass. For forward pass for layer like 1x1 conv or 3x3 conv, i was able to easily compute number of ops/or memory use.
But I'm unclear on how to compute the number of ops and memory use in backward pass (bprop). I'm not using a implemented model (like pytorch) to measure this by running in pytorch, I simply want to understand how to compute this just on the basis of layer type, filter sizes, input output sizes etc.
How to do this? Could someone explain this here? Thanks.