I have a question i am hoping somebody might be able to help with. I have trained a sequential model (keras) that attempts to understand multiple labels in a data set. There are hundreds of samples (more samples to follow), and each individual sample will have multiple labels (an organ, an age and sex).
the model performs really well and is able to understand every concept above (age sex and organ), with really high performance. I would like to know which nodes in the network interact with one another, especially, which samples in the network interact with one another (i.e, are the features that activate for example, a brain and lung if your male at a specific age).
For this reason, I have taken the activation's of every node in the hidden layers and I attach a UMAP to this post. There is a slight problem because the most common activation are actually organ specific. This makes sense as the concept of organ is very strong, so it not surprising that this is the easiest class to pick apart. The problem is is that when I use dimensionality reduction on the node activation, the nodes that separate most closely cluster according to organ, again because this is the class that has an overwhelming signature.
This is even the case in the penultimate layer (image attached)...
My question is, is there a way of identifying and removing the neurons that lead specifically to organ decisions so that i can then repeat the dimensionality reduction in the intermediate layer to look for relationships in the data?
If i plot a normal heatmap i can see where activation's are shared and which samples where they are shared, but i would also like to use dimensionality reduction to view which neurons activate in a similar manner and what samples share features beyond just the overwhelming organ signature.
i am happy to provide code if required!