I'm trying to learn LSTMs and I thought a nice way of doing it would be identifying onset-and-offset of QRS complexes on ECGs.

I have 300 x 200 x 2 numpy array of ECGs (300 ECGs, each of 200 data points, each data point being [x=time,y=voltage]. Sometimes it's 0 padded at the start (like the example below) as ECGs can be of different durations.

I also have labels in the form 300 x 2, 300 labels of [qrs_onset,qrs_offset].

Labelled ECG example

This is an example of an ECG plotted and labels superimposed.

I've done some pre-processing, shuffling the location of the ECG forwards and backwards by ~ 10% of the cycle length. I then feed it into an LSTM model with Python/Keras/Tensorflow backend:

self.model = Sequential()
self.model.add(LSTM(input_length=199, input_dim=2, dropout=0.1, output_dim=64, return_sequences=True))
self.model.add(LSTM(256, dropout=0.1, return_sequences=True))
self.model.add(LSTM(100, dropout=0.1, return_sequences=False))
self.model.compile(loss="mean_squared_error", optimizer="rmsprop")

However, all the LSTM does is fine a location that fits the entire dataset best, and gives that exact location regardless of the ECG fed to it.

Predicted result

It takes about 4000 epochs to get to this point, but at this stage the loss completely plateaus and makes 0 further progress.

It's strange because I thought an LSTM would be great for this task.

Things I have tried:

  1. Reducing the epoch number to a smaller proportion of the dataset to get out of local minima
  2. Adding between 1 and 3 LSTM layers of between 64 and 300 units.
  3. Using the Adam optimiser instead of RMSProp
  4. Adding and removing dropout between 0.0 and 0.2
  5. Adding a 1D Conv layer to try and identify the peaks using spatial information before feeding this into an LSTM

    self.model = Sequential()
    self.model.add(Conv1D(filters=32, kernel_size=3, padding='same', activation='relu'))
        LSTM(300, dropout=0.2, return_sequences=True, input_shape=(None, 300)))
        LSTM(300, dropout=0.2, return_sequences=True, input_shape=(None, 300)))
        LSTM(300, dropout=0.2, return_sequences=False, input_shape=(None, 300)))
    self.model.add(Dense(2, input_dim=300))
    self.model.compile(loss="mean_squared_error", optimizer="rmsprop")

But the model seems completely incapable of learning to identify the QRS complex.

Does anyone have any advice of how I might improve things. I'm expecting to get more data, but actually I'm not sure that will help here as I'm certainly not over-fitting the data I've already got.

Do you think resampling my data series at regular intervals so it's just voltage in a linear time series might help, so my input is 300 x 200 x 1, rather than 300 x 200 x 1? It's not ideal as the sampling frequency changes, but I am willing to try if people think it will help.


Just in case anyone is interested, I made a few changes to my network and now it works beautifully:

Red curve = Probability of being QRS onset

Green curve = Probability of being QRS offset

Dotted lines are maximum probability for each

enter image description here

Basically the changes I made were

  1. I got rid of the time series array, so the input data was just an array of voltage data
  2. I instead made two networks, one for QRS onset and one for QRS offset; this appears to have reduced the tendency for the network to find one of the parameters, and then just place the other based of the average QRS duration.
  3. I changed the task for a regression task to a classification task, so each data point had a probability of being QRS onset (thanks @shimao)
  • $\begingroup$ I've had similar problems and in the end it usually ends up being a problem with input / output while training. $\endgroup$ Sep 27, 2017 at 10:55
  • 2
    $\begingroup$ Would be helpful to show the changes you made in order to make it work. $\endgroup$ Dec 30, 2017 at 15:13
  • $\begingroup$ can you post your changes ? $\endgroup$ Jan 9, 2020 at 7:01

2 Answers 2


I would suggest framing this as a classification problem and outputting 2 softmaxes each with size 300. This usually works better than the continuous output approach you have taken here.

You might expect this approach to work better, because in order for the LSTM to successfully execute the original regression approach, it would have to detect the onset, and then somehow pass down that information several hundred time-steps. In addition, there would probably have to be a counter-like mechanism embedded in the LSTM weights in order to figure out exactly where the deteted onset was. This is all super difficult for an LSTM to learn to do.

Also for that reason, I don't recommend just taking the hidden vector from the last time-step of LSTM and getting the output from that -- instead, try doing something with the full sequence of hidden states (flatten them or something).

  • $\begingroup$ That's a great idea, thank you! Can you explain or give an example of what you mean flattening the hidden states rather than doing what I've done? I don't quite follow how you'd use that as an output and calculate loss etc. ... $\endgroup$
    – James
    Sep 28, 2017 at 7:48
  • $\begingroup$ So if the hidden state has dimension 64 and there are 300 time-steps, instead of taking the last hidden vector, which has size 64, you might want to take all the hidden vectors by setting return_sequences = True. I imagine this will give you something shaped 300x64, which you can then reshape into a 19200 dimensional vector, and then add some linear layers on top of that. Since 19200 is rather large, I suggest that the last LSTM layer have a smaller output dimension (perhaps 8 or so). $\endgroup$
    – shimao
    Sep 28, 2017 at 12:39
  • $\begingroup$ Thank you, I implemented it as demonstrated in my post and it works beautifully $\endgroup$
    – James
    Sep 29, 2017 at 7:53

I tried to do this also and got stuck at exactly the same point. I made my solution available on github (https://github.com/niekverw/Deep-Learning-Based-ECG-Annotator). After reading seq2seq work of semantic segmentation of pictures, I realized that you could apply it to ECG with LSTMs as well to segment the ECG with categorical_crossentropy. This has the advantage (I think) that there is much more for the model to learn, and you don't need >1 model.

model = Sequential()
model.add(Dense(32,W_regularizer=regularizers.l2(l=0.01), input_shape=(seqlength, features)))
model.add(Bidirectional(LSTM(32, return_sequences=True)))#, input_shape=(seqlength, features)) ) ### bidirectional ---><---
model.add(Dense(64, activation='relu',W_regularizer=regularizers.l2(l=0.01)))
model.add(Dense(dimout, activation='softmax'))
adam = optimizers.adam(lr=0.001, beta_1=0.9, beta_2=0.999, epsilon=1e-08, decay=0.0)
model.compile(loss='categorical_crossentropy', optimizer=adam, metrics=['accuracy']) 


  • Colors at the top show true annotations
  • Colors at the bottom show predicted annotations

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.