Let's say we have an Input 10x10x3 (WxHxD) and 5 filters 3x3x3. Convolution between Input and filters will be 8x8x5.
During backpropagation we will get error with the same size 8x8x5. While calculating dE/dF how do I perform convolution when depth between filters and error do not match. The same thing goes to dE/dX.
I understand that I need to perform 'FULL' convolution while calculating dE/dF but that's just impossible without the same depth.