What type of neuron values should we use for similarity functions when comparing two neural network representations?

Question

I asked a related question on how to center the input to similarity functions designed to compare neural networks and was wondering if we should input the neurons after the activations or the neurons after the activations?

The papers and talks I watched/read say "neurons" but never specify that detail. I think that pre-activations is what we should use because if we use the output of activations then some values might be randomly killed and the similarity might be artificially increased (just because of the activation function). But using the pre-activation might mean that if you have square collection of neurons to compare things like CCA which are built to be invariant to invertible affine transformers might not be as good as expected (since we are using the pre-activation values, the values after the fully connected layer).

So what should we use?

Personally, I think I will use the pre-activation values and hope that the NNs I am analyzing are wide enough such that CCA and other metrics do not break.

---

related/refs:

• https://www.youtube.com/watch?v=TBjdvjdS2KM
• https://arxiv.org/abs/1905.00414
• https://arxiv.org/abs/2108.01661

Answer 1

You need to define how you intend to "compare" two neural networks. The most obvious route would be to compare how they encode your input variables into a latent space z, gather samples, estimate a distribution, bada bing. You could also compare them on accuracy. Get my point?