1
$\begingroup$

Suppose we have a standard autoencoder with three layers (i.e. L1 is the input layer, L3 the output layer with #input = #output = 100 and L2 is the hidden layer (50 units)). I know the interesting part of an autoencoder is the hidden part L2. Instead of passing 100 inputs to my supervised model, it will feed it with 50 inputs. What is the optimal hidden units size? 50 is well, but why not using 51, 52 or 63 hidden units? Does 51 will perform better the supervised model than 50 hidden units?

Suppose now that the number of inputs is 1,000,000. If N is the number of units, then I don't want to test out each possible value for N to find out the optimal N. I thought there exists at least an algorithm to do not be obligated to test each possible value or eliminate some of them.

Could that question help?

$\endgroup$
1
$\begingroup$

In short: there is no way to tell the optimal number of units in a hidden layer without actually trying.

There is a number of heuristics, intuitions, rules of thumb, best practices etc, however, number of units in a layer is simply a hyperparameter that requires tuning like any other, and you will simply have to try several values to find the best one. Also, neural networks are usually quite robust, so a few units more or less should not make a big difference.

For more details, see this answer. Also, the thread you linked contains some good hints.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.