Dataset: 40k uniformly distributed 13-dim samples (floats between 0 and 1)
AutoEncoder: (input: 13dim) - fc layer 13 dim, relu - latent layer - fc layer 13 dim, relu - (output: 13dim)
I am using this toy problem to check if I can get a good reconstruction of the input data.
Using a simple AutoEncoder (just reconstruction loss, MSE), if i keep the latent layer of also 13-dim, I get a good reconstruction after 4k epochs.
However, if I reduce the latent dim to 2, I am having problems obtaining a good reconstruction of the input data even after 5k epochs. Visualising the first two dimensions, the reconstruction looks squished w.r.t. the input.
Does anyone have experience with such a setting?
I am not sure if the problem is with the data being uniform or there is another bug I should look into.