2
$\begingroup$

I want to try generate music using LSTMs from MIDI data. The model is based on the prediction of the next notes based on the previous sequence - based on known language models eg. char-rnn. To train I use 24 midi files of Chopin preludes.

  1. I parse midi files into notes notation eg. b d e
  2. Next I must create sequences and notes The output for each input sequence will be the first note.

For example for this plelude X will be sequences of notes and y it's predicted note after sequence.

enter image description here

          X                   Y
['C2', 'G3', 'G2', 'C4', 'E3'] => G4
['G3', 'G2', 'C4', 'E3', 'G4'] => E4
['G2', 'C4', 'E3', 'G4', 'E4'] => C4
['C4', 'E3', 'G4', 'E4', 'C4'] => A4
['E3', 'G4', 'E4', 'C4', 'A4'] => A3
['G4', 'E4', 'C4', 'A4', 'A3'] => B1
['E4', 'C4', 'A4', 'A3', 'B1'] => G3
['C4', 'A4', 'A3', 'B1', 'G3'] => G2
['A4', 'A3', 'B1', 'G3', 'G2'] => D4
['A3', 'B1', 'G3', 'G2', 'D4'] => F3

I put this data into neural network. I train model on NVIDIA Tesla K80 on 4h. The output melody is nice but it's not perfect. The problem appeared when I divided into a test and teaching set. The charts show that the model is overfit. I do not know how to improve it. I have already reduced the number of cells and added a larger dropout. The charts show that my model is overfitted, how can I fix it? It's a problem?

enter image description here This is model architecture.

 model = Sequential()
    model.add(LSTM(
        256,
        input_shape=(network_input.shape[1], network_input.shape[2]),
        return_sequences=True
    ))
    model.add(Dropout(0.3))
    model.add(LSTM(512, return_sequences=True))
    model.add(Dropout(0.3))
    model.add(LSTM(512))
    model.add(Dense(256))
    model.add(Dropout(0.3))
    model.add(Dense(n_vocab))
    model.add(Activation('softmax'))
    model.summary()
    model.compile(loss='categorical_crossentropy', optimizer='rmsprop', metrics=['accuracy'])

    filepath = "weights-improvement-{epoch:02d}-{loss:.4f}-bigger.hdf5"
    checkpoint = ModelCheckpoint(
        filepath,
        monitor='loss',
        verbose=0,
        save_best_only=True,
        mode='min'
    )
    callbacks_list = [checkpoint]

    history = model.fit(network_input, network_output, validation_split=0.33,
                        epochs=600, batch_size=64, callbacks=callbacks_list)
    print(history.history.keys())

    # acc history
    plt.plot(history.history['acc'])
    plt.plot(history.history['val_acc'])
    plt.title('model accuracy')
    plt.ylabel('accuracy')
    plt.xlabel('epoch')
    plt.legend(['train', 'test'], loc='upper left')
    plt.savefig("acc_history.png")
    plt.close()

    plt.plot(history.history['loss'])
    plt.plot(history.history['val_loss'])
    plt.title('model loss')
    plt.ylabel('loss')
    plt.xlabel('epoch')
    plt.legend(['train', 'test'], loc='upper left')
    plt.savefig("history_loss.png")

How can I regularize this? It's possible?

$\endgroup$
1
$\begingroup$

With informations that you give, I would suppose 3 possibilities :

  1. You don't have enough data to generalize, maybe use more data/other preludes depending to your problem

  2. Maybe your training/test set don't have the same distribution, which means you learn a representation which not represent your testing data. Check distributions.

  3. You overtrain your network, use Early stopping : with your data, the best configuration of your network is around 100 epochs.

First I would try to check distributions of my datasets, then increase dropout (to 0.5) and then try to get a larger dataset.

$\endgroup$
  • $\begingroup$ thanks for reply, how can I check distibutions? $\endgroup$ – lukassz Jul 31 '18 at 13:17
  • $\begingroup$ As I can see, you want to predict a note from a sequence, you can check frequency/bayesian inference/pattern recognition of notes from sequences and compare this measures from train and test. $\endgroup$ – alexandre_d Jul 31 '18 at 13:57
  • $\begingroup$ There is some functions in keras for this? $\endgroup$ – lukassz Jul 31 '18 at 14:33
  • $\begingroup$ Unfortunately no, keras don't give analysis tools but you can look to scikit-learn for time series analysis. $\endgroup$ – alexandre_d Jul 31 '18 at 14:39
  • $\begingroup$ Ok I do some research and I will learn the time series because I do not know them $\endgroup$ – lukassz Jul 31 '18 at 14:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.