I try to understand how the multi digit classification in this paper with the Google Street View data works. They try to detect multiple digits within a picture without localization (5 digits and the length of the sequence).
They state that they use 6 classifiers on top of convolution layers:
Each of the variables above is discrete, and when applied to the street number transcription problem, each has a small number of possible values: L has only 7 values (0, . . . , 5, and “more than 5”), and each of the digit variables has 10 possible values. This means it is feasible to represent each of them with a softmax classifier that receives as input features extracted from X by a convolutional neural network
Each of the softmax models (the model for L and each Si) can use exactly the same backprop learning rule as when training an isolated softmax layer, except that a digit classifier softmax model backprops nothing on examples for which that digit is not present.
As far as I understand it this means they use some convolution layers for feature detection and these features are basically the inputs of 6 independent classifiers. And these classifiers (e.g. some fully connected layers) are trained but the output of the conv layers stays fixed (backprop stops at the first fc layer).
But how are the conv layers trained?