2
$\begingroup$

I would like to perform dimensionality reduction using autoencoders (similar to PCA) and I am not sure how many components are optimal i.e. what should be the size of the bottleneck layer.

I was wondering if there is any standard way to automatically find the optimal number of nodes in the bottleneck layer.

$\endgroup$
2
$\begingroup$

No, there is no standard way. In some cases, we pick two or three nodes because it is easier to visualise the resulting representation. Aside that there is little formal work especially when auto-encoder generated representation are use in other applications (classification, denoising, etc.)

In general, the bottleneck layer constrains the amount of information that goes through our auto-encoder, this forces the bottleneck to learn a "good but compressed" representation of our original input data. There has been some work on how bottleneck size affects the overall quality of the embedding which focused in particular application domains: Gupta et al. (2016) "Squeezing bottlenecks: exploring the limits of auto-encoder semantic representation capabilities" focused their attention to text reconstruction. We can potentially recreate scree-like plots for various metrics (e.g. SPI - Structure Preservation Index or SAI - Similarity Accumulation Index) the ones shown in the paper for our particular application.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.