I'd like to create a model capable of emulating music that has been presented to it. The model ought to be specifically designed for that purpose, not just another generic, stacked LSTM. For the purposes of this question we'll only address generating chords, although I'll probably extend the model later if this is even remotely succesful.
Music, unlike many other sequences like text, is about relationships, not so much about absolute values. A problem with basic architectures is that relationships between inputs are not generalised well. So given sequences of (one-hot-encoded) inputs like 1-1-3 and 2-2-4 the model has no clue what comes after 3-3.
I do have an idea of what could be a good representation of a chord and an architecture to support that idea. Chords are agnostic about the pitch or voicing (order of notes) they are played in, and a chord cannot contain two notes that are the same. As information about those things is redundant at best, misleading at worst, we could dispense with that information.
There are a total of twelve notes in an octave, after which the notes repeat. Imagine a ring of those twelve notes. That ring represents every possible note collapsed into one octave (note pitch mod 12). Chords are placed onto that ring.
These qualities would be desirable for a model.
- Learns from sequences, so predictions are heavily influenced by previous steps
- Generalises on intervals (distances between notes), so sequences like 3-4-5, 7-8-9 and even 12-1-2 would be viewed as similar.
In the videos linked above, two architectures had a convolutional element to them. One was fully convolutional (HyperGAN), the other used convolution slightly differently (PixelCNN). I'd imagine some similar construct would be needed here.
Let me write my thoughts down: I'm not sure if a convolutional structure is needed or just something that resembles it a lot. In my mind I'm imagining a node for each interval, such that one node (or layer or heck even a network) would recognise shifts of 1, another shifts of 2, 3 and so on.
As my knowledge and intuition of convolutional and recurrent neural networks is limited to say the least, I'm not sure what would be the best structure to use here. Have you encountered this kind of need before? In music or elsewhere. I'd love any pointers or insight.
Is the structure described above suitable for modelling the problem? Would using straight-out convolutions change the situation, for better or for worse? If convolutions are used, can the gap in the ring of notes between 12 and 1 be bridged?
Why Cross Validated?
This is a design-heavy question, so it could go on Software Engineering, but the nature of the problem - machine learning - convinced me this would be a more appropriate place. The question addesses the high-level design, not the implementation on software level.