Will each unique input to an Autoencoder produce a unique coding? Today I was in the middle of a particularly boring lecture. During this class I briefly let my mind wander. During this time I wondered:

Given a unique, but possibly similar, set of input data is it possible for the encoder to create overlapping codings for two or more different inputs?.

Suppose that I have an arbitrary set of inputs, $X$. In this input set, there are no duplicate data instances, but the data can be very simular. If you were to feed this data into a sufficiently trained autoencoder is it possible for the encoder to output the same codings for multiple different instances of the input set?
 A: In some setups, not only they can, they need to. An idealized Denoising Autoencoder with a weak decoder would map any input+noise, as well as just input, to the same eventual latent code - its encoder would be just a lossless compression of the noiseless data, plus noise filters.
For a negative case, in a pathological scenario the latent encoding could collapse into a single vector, producing a single underfitted reconstruction with a local minimum of reconstruction cost.
That's just classical AEs. A VAE should produce overlapping codes, if you consider the code to be the sample rather than the distribution parameter, being N-dimensional bubbles in a compact and (approximately) continuous latent space.
A: Trivially, if your bottleneck/representation layer uses ReLU activations and all of the inputs to that layer are less than 0, the encoding will be all 0s. So to produce such encodings, you'd just need to have two inputs that have the property that they get mapped "to the left side" of all the bottleneck layer ReLUs.
Or you have an auto-encoder with weights that are all 0 (maybe because you set the $L^2$ regularization too high and the model collapsed), this model assigns all inputs to the same code.
A: There is more information going to the bottleneck than it is going out, so some input have to produce the same outputs.
