For example, in VGG/OxfordNet, the fully connected (dense) layers that precede the final classification layer are of size 4096.
Similarly, in an AlexNet
... the number of neurons in the network’s remaining layers is given by 253,440–186,624–64,896–64,896–43,264–4096–4096–1000.
My question is what is the rationale for using this particular number of neurons in the dense layers?
I haven't found a satisfactory explanation, given that each of the above examples use different resolution images and different number of kernels and kernel sizes, thereby generating different number of parameters—but regardless use similar dense layers.
Even if this is purely empirical, I'd be interested in hearing your thoughts on how this is decided.