Lenet - Limitation on input size? I have been studying Convolutional NNs and in a case study as well as in tutorials, I have noticed that the Lenet is used for handwritten digit classification for reading pin codes etc. Thus , is it true that Lenet works well with only small sized images of 32x32 ? Or is it just, that it was outperformed by AlexNet (and other such NNs) due to more number of layers that work better for more complex problems and that the performance of Lenet has nothing to do with it's input size ?
 A: To answer your questions, I have to give a little bit of background information.
LeNet
LeNet was described in Gradient-based learning applied to document recognition (1998). Please note that this is before GPU programming (CUDA was released in 2007). This means it was practically impossible to train large networks.
LeNet-5 is a CNN (see Figure 2) which works on 28×28×1 input data. LeNet-5 has 7 layers.
LeNet-5 was designed to work on the MNIST dataset.
Alexnet
Alexnet was described in ImageNet Classification with Deep Convolutional
Neural Networks (2012). They use a CNN with ReLU activation functions and 8 layers. One interesting non-standard thing is overlapping pooling areas.They take 224×224×3 images (the 3 is RGB). The last two layer are fully connected with 4096 neurons each. They apply dropout regularization.
Alexnet was designed to work on the Imagenet dataset.
Your questions
LeNet was not designed to work on large images. Just like Alexnet, LeNet is designed to work on a fixed-size input. There are ways to design networks for arbitrary sizes, though (see FCNN). However, often it is easy to adjust the first layer to make the network (in principle) work with different sized input.
Having said that, Alexnet would still perform better than LeNet on big images. (Dropout) regularization is VERY important when it comes to large networks. The training time is faster with ReLU units (I'm not too sure what LeNet uses).
