Updating of Filters in CNN

Question

In normal neural network, in the first layer we have data, random weights and bias term. This then pass through hidden layer, then out output layer, we then calculate the error, and finally based on error we again update the weights.

This looping is done continuously till we get satisfactory error rate.

But in CNN, we first use different filters (say 5 filters) to extract features. After 2 convulation layer and max-pooling plus padding plus flattening, we get a vector. This vector is input vector as we observe in normal NN. We also have random weights and bias terms. This value is then propagated into fully connected layer and follows the same pattern as we observe in normal NN.

My questions are the following:

1) In case of normal NN, the weights and biases are updated during backward propagation. In CNN, apart from weights and biases does filters values are also updated?

2) Related to first question, how the filters are chosen? "Filters are chosen" means types of filters are chosen. As per my understanding different filters extract different features of an image. As well as how number of filters are chosen.

Answer 1

As @Alex R. already explained in his comment, convolution filters are also weights that get updated during the training. There are several ways how the filters are chosen, and there are two aspects to consider: architecture and weight initialization.

Architecture of the convolution filters means how many filters does the network have, how large they should be, how many layers of them. This is mostly driven by current rules of thumb, as I described in this answer.

Initialization of the convolution filters means how should they look at the start of the training. There are two main streams:

1. Random initializations (notably "xavier" initialization from Glorot and Bengio 2010 paper, explained nicely in this blog; and "msra" from the ResNet paper by He et al., 2015.)
2. Transfer learning, domain adaptation (using convolution filters from a network trained on some other task in the same input domain). A list of sources of pre-trained networks can be found in this answer.

A more thorough overview of possible intialization schemes from the perspective of their regularization abilities is in my article Regularization for Deep Learning: A Taxonomy, Section 7.