What features are suitable when predicting user preferences for songs? I have a data set consisting of 1240 audio files (30 seconds each) and a file like this (two first rows):
u  v   decision
1  323 0
12 9   1

u and v are different audio IDs, whereas decision is which one of the songs the user preferred.
I want to build a GLM that can predict the user's decision, based on the input of two songs. However, I am not sure what features to extract from the songs, as I don't have much experience with audio files. As of now, I have converted the audio files to gammatone-based spectrograms. 
My design matrix consists of two gammatone-based spectrograms subtracted from one another (the two songs I am comparing).
Does it make sense? Does anyone know specific features that are relevant for such problems?
 A: Try MFCCs (Mel-Frequency-Cepstrum-Coefficients). They are widely used and you can find a lot of examples to get a good understanding and references about useful parameterisation:
See i.e. the project on spotify content based recommendation http://benanne.github.io/2014/08/05/spotify-cnns.html). 
Gammatone filters are probably suitable too, but they are a lot more complex / harder to explain / supposedly computationally expensive
For further information on audio feature extraction see the online course of Alexander Lerch: https://www.audiocontentanalysis.org/teaching/
Why do you want to subtract the spectrograms? More precisely, why should the amount the spectrograms differ from each other control the outcome of the choice?
Lets substitute music for cake in this experiment. Our participant will be presented with two options of cake. As with the music we have 1240 different recipes. We know the ingredients for each, and we also have some data about past choices of our participant.
If we are lucky, we find i.e. that the participant always took the cake with less sugar, and has high preference for those with a lot of vanilla. 
With that information we can give ratings to all our recipes (low rating for high sugar, high rating for high vanilla). Our prediction for the participants choice now is a mere comparison of which of the two recipes has the higher rating.
If we however subtracted one recipe from the other, this implies something like: "if the cakes largely differ in amount of sugar, participant will take a, otherwise participant will take b". I am not saying that this is not a possible effect, but from my every day experience it will definitly not be the strongest.
In one sentence: Do not subtract the spectrograms, but use them before hands to create song ratings that you can then compare between two given examples.
A: Instead of predicting which features are relevant up front, extract a wide range of features, then use your data to filter out the relevant ones. Since your labels are on the whole song, start with features which are (summarized) per-song.
Essentia implements a huge range of feature extraction algorithms for audio, including many that are specific to music.
Besides doing exploratory data analysis on the features, a RandomForest classifier can be useful starting point, since it can indicate feature importance and handles correlated and noisy features well.
