2
$\begingroup$

I'm running PCA on a image data with 4 components. Obviously I could just multiply the projections with the components to approximately recover my original data set, but I can also view each projection multiplied by it's corresponding component individually. Using this technique I can visualize how much each component contributes to the original image.

Is there a similar technique for autoencoders that will let me see, visually, the contribution each feature has on the input image? Can this technique be extended to autoencoders that use convolutional layers?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

In a deep autoencoder, you can freeze all components of the bottleneck layer and then perturb them one by one to see how each of them affects the output. This may lead to interesting insights. However, as you increase the value of a specific component, do not expect the output to only increase its intensity, because the model is not linear. Also, do not expect a combination of non-zero components to produce an output that is a linear combination of the output each component produces separately. This will work with convolutional autoencoders as well as fully connected autoencoders.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Is there a standard way to perturb the features at the bottleneck? I've tried zeroing out a feature, reconstructing, and subtracting from the original reconstruction to get a sense of a feature's "contribution" but I'm not sure if this is valid. $\endgroup$
    – ayak
    May 17, 2020 at 20:43
  • $\begingroup$ @ayak the reconstruction is not a linear transformation from the bottleneck so there is no point in subtracting from the original reconstruction. All you can do is look at the reconstruction itself for different bottleneck values, which is called to "walk in latent space" (search those words and you will find more info) $\endgroup$
    – elliotp
    May 18, 2020 at 5:46
  • $\begingroup$ Thank you for pointing me in the right direction $\endgroup$
    – ayak
    May 18, 2020 at 19:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.