I’m training a fully convolution network with final layer of global sum pooling and no intermediate pooling in the network which I already test on global average pooling and it converged very well.
The reason I'm testing the global sum is to penalize harder the activation map and make them only sensitive to the actual classification. In my case I have two classes either face or not. By using global average pooling the average prediction is giving 1.2 err but looking at the activation of the counter class I can see that it is also activated on the same local features but with lower value.
Few values to interpret the model:
- learning rate 1e-10 which gave the lower training cost during the first epochs.
- L2 1e-3
- SGD and nestrov tested.
- 6 layers of 7by7 and 3by3 before applying 1by1 to reduce dimension to 2 feature maps.
- first epoch error was 25% whithin 700 epochs error reached 44% while cost around 5.5%
I can see it that the network is getting more confident about some examples in the dataset and overfitting them while loosing it generalization. the solution might be to increase the capacity and complexity of the network
