0
$\begingroup$

this is my first time asking question on CrossValidated, so if there is any mistake on my part, i apologize. I will try not to make those mistakes again.

I'm trying to do a NER task. The problem is the imbalance of the datasets. There are about 98% of 'OTHERS' tags, the rest are NER tags.

I have tried training simple GRUs neural network but it always result in predicting 100% of 'OTHERS' tags.

And the reason there are 98% of "OTHERS" tags is because i padded all the inputs 0 value and padded their corresponding ground truth with "OTHERS" tag.

My model looks like this:

model = keras.Sequential([
    Embedding(len(words_index)+1, 200,  weights=[word_embeddings], trainable=False),

    Dropout(0.1),

    Bidirectional(GRU(128, kernel_initializer='glorot_normal', return_sequences=True, 
                       bias_initializer='zeros', recurrent_dropout=0.1)),

    TimeDistributed(Dense(len(tag2id), activation='softmax'))
])

ops = keras.optimizers.Adam(lr = 0.001)

model.compile(optimizer=ops, loss='sparse_categorical_crossentropy')

I'm a beginner in ML in general and NLP in particular so i guess there is something that i did wrong here.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.