this is my first time asking question on CrossValidated, so if there is any mistake on my part, i apologize. I will try not to make those mistakes again.
I'm trying to do a NER task. The problem is the imbalance of the datasets. There are about 98% of 'OTHERS' tags, the rest are NER tags.
I have tried training simple GRUs neural network but it always result in predicting 100% of 'OTHERS' tags.
And the reason there are 98% of "OTHERS" tags is because i padded all the inputs 0 value and padded their corresponding ground truth with "OTHERS" tag.
My model looks like this:
model = keras.Sequential([ Embedding(len(words_index)+1, 200, weights=[word_embeddings], trainable=False), Dropout(0.1), Bidirectional(GRU(128, kernel_initializer='glorot_normal', return_sequences=True, bias_initializer='zeros', recurrent_dropout=0.1)), TimeDistributed(Dense(len(tag2id), activation='softmax')) ]) ops = keras.optimizers.Adam(lr = 0.001) model.compile(optimizer=ops, loss='sparse_categorical_crossentropy')
I'm a beginner in ML in general and NLP in particular so i guess there is something that i did wrong here.