1
$\begingroup$

I have two question regarding regarding Convolutional Neural Network (with Autoencoders for patches generation). Let's assume that I got a dataset with images and I want to perform an object recognition task using Convolutional Neural Networks. In the first step I am going to implement Autoencoders Network in order to generate the patches. A brief description about Autoencoders could be found here. My first question is concerning this proess. In this step my system could be learn weights W which is in fact is learning an identify function. I am confused on how can I visualize those weights and produce the patches images. Every hidden unit(or simple neuron in hidden layer) is just a value. How can I get the results which are presented in the mentioned link?

My second question is in CNN architecture. Having calculate patches in the first step, in the convolution layer I have to convolve all patches with the image? Thus if I have 100 patches I will create for every input image 100 convolution size-layer? What if I want a second convolution layer? I have to convolve the results of max-pooling with the same patches I have generated in the Autoenconder step?

EDIT: One last question: If this is the architecture of autoenconders:

enter image description here

The visualised weights are those between L2 and L3?

$\endgroup$
  • 1
    $\begingroup$ To your first question, you are using the input weights (normalized by the value in the denominator of the equation for x[j]) as pixel values. $\endgroup$ – John Yetter Jun 17 '15 at 13:18
  • $\begingroup$ Ok I am confused about the size of the weights. Input layer is n= 100 the output layer is y = 100 and for example hidden layer size is 100 thus I got weight in the input layer 100 values and for the output layer 100 values. $\endgroup$ – Jose Ramon Jun 17 '15 at 13:22
  • 1
    $\begingroup$ For any neuron, you can produce a picture that represents what that neuron is triggering on. So, for each neuron, you use its input weights. $\endgroup$ – John Yetter Jun 17 '15 at 13:27
  • $\begingroup$ Ok so for example if I have an image 10x10 thus n=100 I have 100 values for every neuron. In this tutorial ufldl.stanford.edu/tutorial/supervised/… there is a mention that there is a restriction to the connections between the hidden units and the input units, allowing each hidden unit to connect to only a small subset of the input units. For this case the unconnected nodes will have just zero value? $\endgroup$ – Jose Ramon Jun 17 '15 at 13:39
  • $\begingroup$ @JoseRamon I've edited my answer. Please leave a comment when you augment your question, so answerer can see it. $\endgroup$ – yasin.yazici Jun 18 '15 at 18:26
2
$\begingroup$

I see you are following a greedy layer-wise training with autoencoders. An autoencoder learns kernels or filters, in the case of images, which are group of units incoming to the same hidden unit. Hence, activation of each hidden unit indicates presence of a certain pattern. If size of input patches are $ mxm$ and number of hidden units are k, there are k learnt filters. You can visualize each kernel by reshaping weight vector incoming to a certain hidden unit. If weight vector is denoted by $ W_{kj}$, than each row of $ W_{kj}$ corresponds to an unrolled kernel which has size $ 1x(m^2)$. If you reshape it by $ mxm$, it turns into interpretable patterns. Function for visualization, 'display_network' is already provided in the site you mentioned.

It couldn't really understand your second question. I think it is becacause of what you mean by pacthes. Patches are randomly cropped $ mxm$ parts of images. You are not using these patches during convolution layer, but rather using learnt kernels(filters) that I mentioned previously. After convolving each k kernels with an input image, k feature maps with size $ (n-m+1)x(n-m+1)$ are produced with a 'valid' convolution if the input size is $ nxn$. Than each feature map is passed from an activation function. You can't use the same features in the second layer. Second layer feature should be learned similar to first layer by using activations from the first layer.

After convolution/pooling layers, the last feature maps are unrolled into a vector to feed into next fully connected layer. If the last feeature maps are 32@5x5, then the size of the vector is $ 800x1$ for each image. If there are m input images the overall vector is $ 800xm$.

EDIT: The first part of the autoencoder is called encoder and the second part is decoder. The second part is used to recostruct the input image, so it is not used after feature learning. If you stack learnt autoencoders only encoder part remains. So, what you are looking for is the first part. The weights you want to visualize is between L1 and L2 layers.

$\endgroup$
  • $\begingroup$ When mentioned patches I meant kernel filters learnt by the autoencoder step. Therefore if I produce k filters in autoenconder step then in the convolution layer I ll have k features maps. Lets say I have 1 convolutional layer only and one max -pool layer. Finally I will have k feature maps. How can I produce from those feature maps a vector from my image? $\endgroup$ – Jose Ramon Jun 18 '15 at 9:56
  • $\begingroup$ @JoseRamon What do you mean by 'How can I produce from those feature maps a vector from my image?'? $\endgroup$ – yasin.yazici Jun 18 '15 at 10:00
  • $\begingroup$ The final step as I am understanding of the CNN is to produce for every image a vector in order to perform a fully connected layer. This is happening for every feature maps separately or there is a merging of all features maps? $\endgroup$ – Jose Ramon Jun 18 '15 at 10:02
  • $\begingroup$ @JoseRamon I've edited my answer. You sould unroll each feature maps and merge them into a single vector. $\endgroup$ – yasin.yazici Jun 18 '15 at 10:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.