I was reading the lecture notes for sparse auto-encoders from Andrew Ng and was saw that it had very nice compact way of expressing back propagation for neural networks:

enter image description here

The really nice thing about this explanation is that its compactly expressed using operations with vectors and matrix multiplication (and dot products and the Hadamard product $\cdot$, that simply denotes the element-wise product operator). However, I was wondering if there was a way to similarly express the above equations for convolutional neural nets but instead of matrix multiplication with the convolution $*$ operator.

I think that its possible to re-derive it, however, since CNNs are so popular these days and they seem to be widely used, it seemed reasonable to assume that this really wasn't an open problem and that there ought to be something like this.



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.