2
$\begingroup$

Imagine the following scenario:

  1. Given a signal x, you pass it through a bank of convolutional filters to get a feature map z (the actual operation is a cross-correlation).

  2. Then you apply a nonlinearity to (potentially sparsify) the features, and

  3. deconvolve using the exact same filter bank, to reconstruct x' with the objective of finding a filter bank such that the difference between x' and x by your distance metric of choice is minimized.

This corresponds to a very basic convolutional (sparse) autoencoder with tied weights.

In this context, the paper Convolutional Sparse Autoencoders for Image Classification (Luo, Yang, Zang) says the following in the introduction:

The weights of the encoder and decoder can be tied to force features to diverge as much as possible.

Further in the paper, they build up on that idea. I can understand the intuition behind that (coding efficiency, model capacity, etc), but I am interested in a more solid mathematical explanation for that. Unfortunately I couldn't find any references in the paper itself. Could you provide such references, and optionally a proof for that? You may assume that the objective is pursued via any form of Gradient Descent, Basis Pursuit, or similar.

Thanks in advance!
Andres

$\endgroup$

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.