# What does a bottleneck layer mean in neural networks?

I was reading the FaceNet paper and in the 3rd paragraph of the introduction it says:

Previous face recognition approaches based on deep networks use a classification layer trained over a set of known face identities and then take an intermediate bottleneck layer as a representation used to generalize recognition beyond the set of identities used in training.

I was wondering what they mean by an intermediate bottleneck layer?

• it is used for reduce the count of calculation, and make a transform of the previous layer. – kxm silence Sep 29 '17 at 5:01

My understanding of the quote is that previous approaches use a deep network to classify faces. They then take the first several layers of this network, from the input up to some intermediate layer (say, the $k$th layer, containing $n_k$ nodes). This subnetwork implements a mapping from the input space to an $n_k$-dimensional vector space. The $k$th layer is a bottleneck layer, so the vector of activations of nodes in the $k$th layer gives a lower dimensional representation of the input. The original network can't be used to classify new identities, on which it wasn't trained. But, the $k$th layer may provide a good representation of faces in general. So, to learn new identities, new classifier layers can be stacked on top of the $k$th layer and trained. Or, the new training data can be fed through the subnetwork to obtain representations from the $k$th layer, and these representations can be fed to some other classifier.