this is my first time asking question on CrossValidated, so if there is any mistake on my part, i apologize. I will try not to make those mistakes again.

I'm trying to do a NER task. The problem is the imbalance of the datasets. There are about 98% of 'OTHERS' tags, the rest are NER tags.

I have tried training simple GRUs neural network but it always result in predicting 100% of 'OTHERS' tags.

And the reason there are 98% of "OTHERS" tags is because i padded all the inputs 0 value and padded their corresponding ground truth with "OTHERS" tag.

My model looks like this:

model = keras.Sequential([
    Embedding(len(words_index)+1, 200,  weights=[word_embeddings], trainable=False),


    Bidirectional(GRU(128, kernel_initializer='glorot_normal', return_sequences=True, 
                       bias_initializer='zeros', recurrent_dropout=0.1)),

    TimeDistributed(Dense(len(tag2id), activation='softmax'))

ops = keras.optimizers.Adam(lr = 0.001)

model.compile(optimizer=ops, loss='sparse_categorical_crossentropy')

I'm a beginner in ML in general and NLP in particular so i guess there is something that i did wrong here.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.