For which phenomenon Convolution Neural Networks work, and for which doesn't and where is a border?

If it means some type of convergence then in which sense - pointwise, uniform, in L2 norm.

CNN as I understand mean "closure" os such things:

  • Input variables

  • Compoistion OR system cascading OR functional graph, use term which you prefer

  • "Convolution node", which is in fact do discrete convolution with some small kernel (3x3) and do some extra bias.

  • Max pooling operation – take a maximum value of blocks 2x2

  • "ReLU" block. I don't know why it was called so, but it evaluates value of “ramp” function


1 Answer 1


I'd expect that a sufficiently large convnet could compute any function of the input pixels. Here's an informal "proof":

  • A convolutional layer can pass through all the information in the previous layer, e.g. by using the identity function.

  • Once the information from the original image has been propagated through the convolutional layers to a set of fully-connected layers, we're left with a generic multilayer perceptron, and we can use existing proofs to take us the rest of the way.

The convergence guarantees are probably quite weak, for the same reason that they are difficult in the case of other neural networks. I don't see why they should be any weaker than the general case, though.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.