4
$\begingroup$

I'm having a problem in my Deep Learning model where I encounter a class imbalance and there's no virtual difference between the data for the two classes or they have an identical if not similar distribution.

One way I tried to solve the class imbalance is through resampling to create an evenly distributed training set. However, my model only predicts one class. This is an NLP problem and I am not sure of using techniques like XGBoost to NLP / Deep Learning problems.

I've read this paper that recommends SMOTE and from this post, am interested in Random Forest, but am not sure of its applicability to NLP. From this paper, there are some algorithms I could use and a different loss function I could provide to the algorithm, but I am using a pre-trained model and so am constrained.

My main problem however, is that there may not be an observable difference between the data in the two classes. In general, can ML / DL work if there is no identifiable difference between the data across the two classes?

Is there any way I can try and manually find some difference between the data for the two classes?

$\endgroup$

1 Answer 1

10
$\begingroup$

No machine learning method can (reliably) separate classes that are inseparable. Let’s consider two example.

In this first case, we have data generated by $N(0,1)$ with label $0$ and data generated by $N(2,1)$ with label $1$. You might be out of luck when it comes to predicting the label for an observed value of $1$ (equally likely to have come from either distribution), but for something like $4$ or $-2$, you have a pretty good chance of classifying those correctly.

Now consider data generated from $N(0,1)$ with label $0$ and more data generated from $N(0,1)$ with label $1$. Given an observation, do you have any chance of identifying the correct label?

It turns out that you don’t have different classes. Separation is impossible.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.