# Generate music using LSTM using language model overfitting problem

I want to try generate music using LSTMs from MIDI data. The model is based on the prediction of the next notes based on the previous sequence - based on known language models eg. char-rnn. To train I use 24 midi files of Chopin preludes.

1. I parse midi files into notes notation eg. b d e
2. Next I must create sequences and notes The output for each input sequence will be the first note.

For example for this plelude X will be sequences of notes and y it's predicted note after sequence.

          X                   Y
['C2', 'G3', 'G2', 'C4', 'E3'] => G4
['G3', 'G2', 'C4', 'E3', 'G4'] => E4
['G2', 'C4', 'E3', 'G4', 'E4'] => C4
['C4', 'E3', 'G4', 'E4', 'C4'] => A4
['E3', 'G4', 'E4', 'C4', 'A4'] => A3
['G4', 'E4', 'C4', 'A4', 'A3'] => B1
['E4', 'C4', 'A4', 'A3', 'B1'] => G3
['C4', 'A4', 'A3', 'B1', 'G3'] => G2
['A4', 'A3', 'B1', 'G3', 'G2'] => D4
['A3', 'B1', 'G3', 'G2', 'D4'] => F3


I put this data into neural network. I train model on NVIDIA Tesla K80 on 4h. The output melody is nice but it's not perfect. The problem appeared when I divided into a test and teaching set. The charts show that the model is overfit. I do not know how to improve it. I have already reduced the number of cells and added a larger dropout. The charts show that my model is overfitted, how can I fix it? It's a problem?

This is model architecture.

 model = Sequential()
256,
input_shape=(network_input.shape[1], network_input.shape[2]),
return_sequences=True
))
model.summary()
model.compile(loss='categorical_crossentropy', optimizer='rmsprop', metrics=['accuracy'])

filepath = "weights-improvement-{epoch:02d}-{loss:.4f}-bigger.hdf5"
checkpoint = ModelCheckpoint(
filepath,
monitor='loss',
verbose=0,
save_best_only=True,
mode='min'
)
callbacks_list = [checkpoint]

history = model.fit(network_input, network_output, validation_split=0.33,
epochs=600, batch_size=64, callbacks=callbacks_list)
print(history.history.keys())

# acc history
plt.plot(history.history['acc'])
plt.plot(history.history['val_acc'])
plt.title('model accuracy')
plt.ylabel('accuracy')
plt.xlabel('epoch')
plt.legend(['train', 'test'], loc='upper left')
plt.savefig("acc_history.png")
plt.close()

plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.title('model loss')
plt.ylabel('loss')
plt.xlabel('epoch')
plt.legend(['train', 'test'], loc='upper left')
plt.savefig("history_loss.png")


How can I regularize this? It's possible?

With informations that you give, I would suppose 3 possibilities :

1. You don't have enough data to generalize, maybe use more data/other preludes depending to your problem

2. Maybe your training/test set don't have the same distribution, which means you learn a representation which not represent your testing data. Check distributions.

3. You overtrain your network, use Early stopping : with your data, the best configuration of your network is around 100 epochs.

First I would try to check distributions of my datasets, then increase dropout (to 0.5) and then try to get a larger dataset.

• thanks for reply, how can I check distibutions? – lukassz Jul 31 '18 at 13:17
• As I can see, you want to predict a note from a sequence, you can check frequency/bayesian inference/pattern recognition of notes from sequences and compare this measures from train and test. – alexandre_d Jul 31 '18 at 13:57
• There is some functions in keras for this? – lukassz Jul 31 '18 at 14:33
• Unfortunately no, keras don't give analysis tools but you can look to scikit-learn for time series analysis. – alexandre_d Jul 31 '18 at 14:39
• Ok I do some research and I will learn the time series because I do not know them – lukassz Jul 31 '18 at 14:50