The title question seems a bit too vague but actually I have in my mind a very precise problem.
Let's say I present to a neural network an image: an object in a canvas. I vary the image only across 2 dimensions: size and position of the object in the canvas (let's say I use only one object).
I want to check whether there are some neurons selectively "coding" for size/position. E.g. neuron x is not active for size=10px, quite active for size=50px and very active for size=100px. However, changing object's position does not affect neuron x. Instead, it affects neuron y.
I understand that not all neurons are going to be this selective. Most likely, they are going to have a distributed representation and coding a bit of both. I would like to have a measure of that. Let's say a measure of "selective" coding.
Assume that I can get many samples across both dimensions. I am talking about a pre-trained convolutional network with Swish activation function (this can be changed) and with N fully connected layers at the end, but I want to perform the analysis only on the units of the fully connected layers.