# NER with GRUs neural network with imbalance dataset

this is my first time asking question on CrossValidated, so if there is any mistake on my part, i apologize. I will try not to make those mistakes again.

I'm trying to do a NER task. The problem is the imbalance of the datasets. There are about 98% of 'OTHERS' tags, the rest are NER tags.

I have tried training simple GRUs neural network but it always result in predicting 100% of 'OTHERS' tags.

And the reason there are 98% of "OTHERS" tags is because i padded all the inputs 0 value and padded their corresponding ground truth with "OTHERS" tag.

My model looks like this:

model = keras.Sequential([
Embedding(len(words_index)+1, 200,  weights=[word_embeddings], trainable=False),

Dropout(0.1),

Bidirectional(GRU(128, kernel_initializer='glorot_normal', return_sequences=True,
bias_initializer='zeros', recurrent_dropout=0.1)),

TimeDistributed(Dense(len(tag2id), activation='softmax'))
])