1
$\begingroup$

I use a BiLSTM-CRF architecture to assign some labels to a sequence of the sentences in a paper. We have 150 papers each of which contains 380 sentences and each sentence is represented by a double array with size 11 in range (0,1) and the number of class labels is 11.

input = Input(shape=(None,11))
mask = Masking (mask_value=0)(input)
lstm = Bidirectional(LSTM(50, return_sequences=True))(mask)
lstm = Dropout(0.3)(lstm)
lstm= TimeDistributed(Dense(50, activation="relu"))(lstm)
crf = CRF(11 , sparse_target=False , learn_mode='join')  # CRF layer
out = crf(lstm)
model = Model(input, out)
model.summary()
model.compile('adam', loss=crf.loss_function, metrics=[crf.accuracy])

I use keras-contrib package to implement CRF layer. CRF layer has two learning modes: join mode and marginal mode. I know that join mode is a real CRF that uses viterbi algorithm to predict the best path. While, marginal mode is not a real CRF that uses categorical-crossentropy for computing loss function. When I use marginal mode, the output is as follows:

Epoch 4/250:  - 6s - loss: 1.2289 - acc: 0.5657 - val_loss: 1.3459 - val_acc: 0.5262

But, in the join mode, the value of loss function is nan:

Epoch 2/250 :  - 5s - loss: nan - acc: 0.1880 - val_loss: nan - val_acc: 0.2120

I do not understand why this happens and would be grateful to anybody who could give a hint.

$\endgroup$
  • $\begingroup$ i have this issue as well. $\endgroup$ – Nathan McCoy Feb 14 at 14:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.