3
$\begingroup$

I'm trying to reproduce the results from this paper: Mahendran, Aravindh, and Andrea Vedaldi. "Understanding deep image representations by inverting them." arXiv preprint arXiv:1412.0035 (2014).

One basic idea of the paper is to extract visual Information via Gradient Descent from a white noise image. Basically for a given layer and given image, a loss function is minimized by changing the white noise image via Gradient Descent until the layer activations match.
I'm not able to get anywhere close to the results of the paper.

A simple example of what I am doing (for the first convolution layer):
Suppose the following is given: $Im=[1,2,-1]$, weight matrix $W=\begin{bmatrix} -1 &2 \\ 1&1\end{bmatrix}$.
The convolution is performed via matrix multiplication, therefore we use the function $im2col$ to reshape $Im$. $im2col(Im)=\begin{bmatrix} 1 &2 \\ 2&-1\end{bmatrix}$, the convolution is $W*im2col(Im)=\begin{bmatrix} 3 &-4 \\ 3&1\end{bmatrix}$.
Let the white noise $WN_1=[-1,0,1]$. Then $W*im2col(WN_1)=\begin{bmatrix} 1 &2 \\ -1&1\end{bmatrix}$.
Let the loss function $L=1/2*\sum (x_i-y_i)^2$, where $x$ is the result of the convolution of the white noise and $y$ the result for the image.
Backpropagation then leads to the gradient $G=im2colBack(W^T*\begin{bmatrix} -2 &6 \\ -4&0\end{bmatrix})=[-2,-14,12]$.
We can now use this result to update $WN$ via $WN_1=WN-h*G$, where $h$ is some stepsize.
We can do this many times and only calculate $WN_{i+1}=WN_i-h*G_i$ if the loss decreases (here we also increase $h$ slightly), if the loss increases we decrease $h$ and update $WN_i=WN_{i-1}-h*G_{i-1}$.

Is the above approach correct?
Like I already wrote, I'm not able to reproduce the results of the paper. It seems like my Gradient Descent is always reaching a local minima after a few steps. I tried several Images and diffrent stepsizes. Am I missing something?
Is there a way to avoid local minima?

Thanks!

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.