I am training a model, I am trying to lower the training loss. While testing different architectures I increased the number of filters to 128 from 64 - this reduced the training loss. I do not understand why this change happened, the relationship between training loss and filter amount is unclear to me.
How can the amount of filter improve the performance of the model?
Increasing the number of filters increases the capacity of the model, which allows it to fit to the training data better.
Of course, the test time performance (what we care about) is not necessarily better.
More Parameters
Refer to my drawings here and note that one filter results in four neurons in the hidden layer and, thus, four weights. Two filters, following my logic at the link, results in eight neurons in the hidden layer, so eight weights. Three filters results in twelve weights, etc.
As you increase the number of parameters, you increase the model flexibility and the ability to fit complex functions between your feature space (the image pixels, most likely) and the outcome (such as a category). This sounds appealing, until you consider the possibility of that flexibility modeling noise and overfitting, so extreme flexibility need not be fantastic.
MY DRAWINGS FROM THE LINK
Apply the filter to the upper left 2x2 array.
Apply the filter to the upper right 2x2 array.
Apply the filter to the bottom left 2x2 array.
Apply the filter to the bottom right 2x2 array.
Here is the entire layer, with the 3x3 input image mapping to four neurons for the four positions in the image where convolution occurs.
