Generating sequences of musical chords I'd like to create a model capable of emulating music that has been presented to it. The model ought to be specifically designed for that purpose, not just another generic, stacked LSTM. For the purposes of this question we'll only address generating chords, although I'll probably extend the model later if this is even remotely succesful.
Background
Music, unlike many other sequences like text, is about relationships, not so much about absolute values. A problem with basic architectures is that relationships between inputs are not generalised well. So given sequences of (one-hot-encoded) inputs like 1-1-3 and 2-2-4 the model has no clue what comes after 3-3.
These two videos discuss different types of neural networks for music generation. [1, 2]
Initial idea
I do have an idea of what could be a good representation of a chord and an architecture to support that idea. Chords are agnostic about the pitch or voicing (order of notes) they are played in, and a chord cannot contain two notes that are the same. As information about those things is redundant at best, misleading at worst, we could dispense with that information.
There are a total of twelve notes in an octave, after which the notes repeat. Imagine a ring of those twelve notes. That ring represents every possible note collapsed into one octave (note pitch mod 12). Chords are placed onto that ring.
These qualities would be desirable for a model.


*

*Learns from sequences, so predictions are heavily influenced by previous steps

*Generalises on intervals (distances between notes), so sequences like 3-4-5, 7-8-9 and even 12-1-2 would be viewed as similar.


In the videos linked above, two architectures had a convolutional element to them. One was fully convolutional (HyperGAN), the other used convolution slightly differently (PixelCNN). I'd imagine some similar construct would be needed here.
Let me write my thoughts down: I'm not sure if a convolutional structure is needed or just something that resembles it a lot. In my mind I'm imagining a node for each interval, such that one node (or layer or heck even a network) would recognise shifts of 1, another shifts of 2, 3 and so on.
Discussion
As my knowledge and intuition of convolutional and recurrent neural networks is limited to say the least, I'm not sure what would be the best structure to use here. Have you encountered this kind of need before? In music or elsewhere. I'd love any pointers or insight.
Is the structure described above suitable for modelling the problem? Would using straight-out convolutions change the situation, for better or for worse? If convolutions are used, can the gap in the ring of notes between 12 and 1 be bridged?

Why Cross Validated?
This is a design-heavy question, so it could go on Software Engineering, but the nature of the problem - machine learning - convinced me this would be a more appropriate place. The question addesses the high-level design, not the implementation on software level.
 A: I think you have two options:


*

*Keep a representation of chords that is able to reproduce the complexities that you desire, for example several chords at the same time, such as the *kern representation or midi. These representations have the advantage that some complexities can be included such as multiple instruments, note durations and (for midi) even note intensities. The problem can then again be formulated as a time-series. A *kern file is in fact simply a text file that can be transformed into a sequence of symbols.

*Transform all your notes to an array of on or off notes, as you suggest. There is no reason why you can't model this as a time-series in the sense that each timestep would simply have an entire embedding as an input that could be fed to your model (eg 1D convolutional layer or LSTM). The problem here is that, depending on the complexity of the music you would like to represent, you would probably need to implement the overhead yourself that adds the relevant  information about note duration/instrument/etc and append it to your embedding as well.

