0
$\begingroup$

Deep auto-encoders has been successfully used for problems like digit classification. In this example,

enter image description here

images with 28x28 pixels have been classified into digit 0-9. And the feature vector size is 28x28x1 = 784x1. For my understanding, the CNN based deep learning do require the image size to be relatively 'large' because the smallest sliding window can be 3x3, and if the image or image patch is very small, the sliding window will not work.

I am wondering if deep auto-encoders can cope with image with very small size. For example, if we have digit images with 8x8 pixels instead of 28x28 pixels, will us be able to train the deep auto-encoders? And is there a minimum image size for successful deep auto-encoders?

Thanks. A.

$\endgroup$
1
$\begingroup$

There is not really a minimum size. If the input is small enough you won't need to use a convolutional neural network. You can make a deep purely feed-forward auto-encoder. (Sometimes feed-forward networks are referred to as fully-connected convolutional layers.)

$\endgroup$
  • $\begingroup$ For the Matlab example of digit classification (uk.mathworks.com/help/nnet/examples/…), they have 28x28 images and used 100 hidden layers for the first autoencoder, and decreased the size of the hidden representation to 50 in the second autoencoder. They mentioned 'it is a good idea to make the hidden layer size smaller than the input size', but didn't say why. If I used 8x8 images, can I still use 100 and 50 for two autoencoders? Because 8x8=64<100, should I choose smaller hidden layer size like 50 and 25? Thanks. $\endgroup$ – Samo Jerom Nov 3 '16 at 10:21
  • $\begingroup$ The idea of an auto-encoder is to find a more compact encoding of the input. If the hidden layer is bigger than the input then you can just use the input as it's own encoding. No need for a hidden layer at that point. $\endgroup$ – Aaron Nov 3 '16 at 13:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.