# loss function in CRF keras-contrib returns Nan in join mode

I use a BiLSTM-CRF architecture to assign some labels to a sequence of the sentences in a paper. We have 150 papers each of which contains 380 sentences and each sentence is represented by a double array with size 11 in range (0,1) and the number of class labels is 11.

input = Input(shape=(None,11))
lstm = Dropout(0.3)(lstm)
lstm= TimeDistributed(Dense(50, activation="relu"))(lstm)
crf = CRF(11 , sparse_target=False , learn_mode='join')  # CRF layer
out = crf(lstm)
model = Model(input, out)
model.summary()


I use keras-contrib package to implement CRF layer. CRF layer has two learning modes: join mode and marginal mode. I know that join mode is a real CRF that uses viterbi algorithm to predict the best path. While, marginal mode is not a real CRF that uses categorical-crossentropy for computing loss function. When I use marginal mode, the output is as follows:

Epoch 4/250:  - 6s - loss: 1.2289 - acc: 0.5657 - val_loss: 1.3459 - val_acc: 0.5262


But, in the join mode, the value of loss function is nan:

Epoch 2/250 :  - 5s - loss: nan - acc: 0.1880 - val_loss: nan - val_acc: 0.2120


I do not understand why this happens and would be grateful to anybody who could give a hint.

• i have this issue as well. – Nathan McCoy Feb 14 at 14:59