Back propagation in Convolutional neural networks I am trying to understand how CNN works. I want to use them in object recognition task. I thouhgt that CNN is unsupervised networks. My main question is how can I implement the back propagation without using image targets (pre-defined labels), in order to calculate the error function which will affect the weights of the network.
 A: The derivation involves considerations for the fact that CNNs use weight sharing as opposed to feed-forward networks. During both forward and back-propagation you will use convolutions where the weights and the activations will be the functions in the convolution equation.
The pooling layers do not do any learning themselves. Their function is to progressively reduce the spatial size of the representation to reduce the amount of parameters and computation in the network.
During forward propagation, a P by P pooling block is reduced to a single value i.e. value of the “winning unit”. To keep track of the “winning unit” its index noted during the forward pass and used for gradient routing during backpropagation.
During backpropagation, the gradients in the convolutional layers are calculated and the backward pass to the pooling layer then involves assigning the “winning unit” the gradient value from the convolutional layer as the index was noted prior during the forward pass.
Gradient routing is done in the following ways:
Max-pooling - the error is just assigned to where it comes from - the “winning unit” because other units in the previous layer’s pooling blocks did not contribute to it hence all the other assigned values of zero
Average pooling - the error is multiplied by 1 / (P by P) and assigned to the whole pooling block (all units get this same value).
Read a more comprehensive breakdown on the whole backpropagation procedure here
A: CNN are typically supervised.  So you need training data, and you forward propagate the training images through the network, then back propagate the training labels, to update the weights.
There are unsupervised neural networks, for example Geoffrey Hinton's stacked boltmann machines, creating deep belief networks.  You can also stack auto-encoders, which can also perform unsupervised learning.
In both cases, the goal of the unsupervised learning is that the result of forward propagation followed by backward propagation is the original image.
A: Backpropagation is basically a clever way of expressing and calculating the gradient of a cost function, which depends on known target labels and its corresponding inputs. That is the original formulation of CNNs. They are step beyond traditional MLPs, where features are not designed, but automatically learned from data, so that the recognition rate is maximized (with respect to known labels).
If I understood you question properly, I would reformulate it like: is it possible to train a discriminative model without labels (just data, or partially labelled data?).
What I networks does is defined by the loss (or energy) function it minimizes. What sort of gradient based optimization technique you apply, depends on that energy function. This topic is analyzed in detail in the following references:


*

*Loss Functions for Discriminative Training of Energy-Based Models

*A Tutorial on Energy-Based Learning
