2
$\begingroup$

Using PCA, I can capture about 95% of the variance of my data using 3 modes. If I wanted to make an autoencoder for my data would this mean that the size of my latent space should be $1 \times 1 \times 3$? I plot my data using the 3 modes against time and I'd like to be able to do the same with the latent representation of my data.

If it is the case that my latent space needs to be $1 \times 1 \times 3$, will the bottleneck be too small to get an accurate reconstruction?

$\endgroup$
5
  • $\begingroup$ The sentence "If it is the case that my latent space needs to be 1×1×3, will the bottleneck be too small to get an accurate reconstruction?" seems to be circular. If the latent space needs to be a certain size (to get a good reconstruction), doesn't that imply that size is the right size to get a "good enough" reconstruction? $\endgroup$ – Sycorax Apr 22 '20 at 21:59
  • $\begingroup$ In any case, the answer to your question seems to be contained in the answers here: a linear auto-encoder creates a reconstruction which spans the first $k$ PCs. stats.stackexchange.com/questions/120080/… $\endgroup$ – Sycorax Apr 22 '20 at 22:02
  • $\begingroup$ @SycoraxsaysReinstateMonica but is this also the case for nonlinear autoencoders? $\endgroup$ – ayak Apr 22 '20 at 23:30
  • $\begingroup$ No. Nonlinear auto-encoders don't have an obvious relationship to linear models because they are nonlinear. $\endgroup$ – Sycorax Apr 22 '20 at 23:37
  • $\begingroup$ So that's what I'm wondering in my question. Would the latent representation of a nonlinear autoencoder give me an idea of a mode that I could use the same way I do with PCA? $\endgroup$ – ayak Apr 22 '20 at 23:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.