1
$\begingroup$

Deep auto-encoders has been successfully used for problems like digit classification. In this example,

enter image description here

images with 28x28 pixels have been classified into digit 0-9. And the feature vector size is 28x28x1 = 784x1. For my understanding, the CNN based deep learning do require the image size to be relatively 'large' because the smallest sliding window can be 3x3, and if the image or image patch is very small, the sliding window will not work.

I am wondering if deep auto-encoders can cope with image with very small size. For example, if we have digit images with 8x8 pixels instead of 28x28 pixels, will us be able to train the deep auto-encoders? And is there a minimum image size for successful deep auto-encoders?

Thanks. A.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

There is not really a minimum size. If the input is small enough you won't need to use a convolutional neural network. You can make a deep purely feed-forward auto-encoder. (Sometimes feed-forward networks are referred to as fully-connected convolutional layers.)

Improve this answer
$\endgroup$
2
  • $\begingroup$ For the Matlab example of digit classification (uk.mathworks.com/help/nnet/examples/…), they have 28x28 images and used 100 hidden layers for the first autoencoder, and decreased the size of the hidden representation to 50 in the second autoencoder. They mentioned 'it is a good idea to make the hidden layer size smaller than the input size', but didn't say why. If I used 8x8 images, can I still use 100 and 50 for two autoencoders? Because 8x8=64<100, should I choose smaller hidden layer size like 50 and 25? Thanks. $\endgroup$
    – Samo Jerom
    Commented Nov 3, 2016 at 10:21
  • $\begingroup$ The idea of an auto-encoder is to find a more compact encoding of the input. If the hidden layer is bigger than the input then you can just use the input as it's own encoding. No need for a hidden layer at that point. $\endgroup$
    – Aaron
    Commented Nov 3, 2016 at 13:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.