Try MFCCs (Mel-Frequency-Cepstrum-Coefficients). They are widely used and you can find a lot of examples to get a good understanding and references about useful parameterisation:
See i.e. the project on spotify content based recommendation http://benanne.github.io/2014/08/05/spotify-cnns.html).
Gammatone filters are probably suitable too, but they are a lot more complex / harder to explain / supposedly computationally expensive
For further information on audio feature extraction see the online course of Alexander Lerch: https://www.audiocontentanalysis.org/teaching/
Why do you want to subtract the spectrograms? More precisely, why should the amount the spectrograms differ from each other control the outcome of the choice?
Lets substitute music for cake in this experiment. Our participant will be presented with two options of cake. As with the music we have 1240 different recipes. We know the ingredients for each, and we also have some data about past choices of our participant.
If we are lucky, we find i.e. that the participant always took the cake with less sugar, and has high preference for those with a lot of vanilla.
With that information we can give ratings to all our recipes (low rating for high sugar, high rating for high vanilla). Our prediction for the participants choice now is a mere comparison of which of the two recipes has the higher rating.
If we however subtracted one recipe from the other, this implies something like: "if the cakes largely differ in amount of sugar, participant will take a, otherwise participant will take b". I am not saying that this is not a possible effect, but from my every day experience it will definitly not be the strongest.
In one sentence: Do not subtract the spectrograms, but use them before hands to create song ratings that you can then compare between two given examples.