Can imbalance data create overfitting?

Question

I am doing human activity recognition project. I have total of 12 classes. The class distribution look like this:

$\color{red}{If \ you \ watch \ carefully, you \ can \ see \ that \ I \ have \ no \ data \ points \ for \ class \ 11 \ and \ class \ 8.}$ Also, the dataset is highly imbalanced. So, I took minimum data points (in this case 2028) for all of the classes. Now my balanced data look like this:

After doing this it looks like a balance data. $\color{red}{But \ still, \ I \ think \ it \ not, \ because \ I \ have \ zero \ datapoints \ for \ class \ 11 \ and \ class \ 8}$. In my opinion the classes are still imbalance.

I am using CNN model to solve this activity project. My model summary is following:

The main problem is, my model starts overfitting heavily when I train it.

Is it due to my imbalance data( class 8 and 11 has zero data points) or something else?

$\textbf{Hyperperameter:}$

$\textbf{features:}$ X, Y, Z of mobile accelerometer

$\textbf{frame size:}$ 80

$\textbf{optimizer:}$ Adam, $\textbf{Learning rate:}$ 0.001

$\textbf{Loss:}$ Sparse categorical cross-entropy

Answer 1

I think it's easier to separate this in two problems:

1. how to handle imbalanced data ?
2. how to prevent overfitting ?

On the first point there are several solutions which depends on the use case:

• keep imbalanced data: the learner knows the class distribution so it knows that class 5 is more likely than class 9. It get the best accuracy on the complete dataset but class 5 will have a great accuracy and 9 will get a poor accuracy
• rebalance data: this is what you did, errors balancing errors between classes. This is good for anomaly detection for instance. Many methods exists for this in the literature.

On the second point, ovefitting means that the model trained on training data don't generalize well on test data. It may sound obvious, but it means that this can come from the model, but can also come from training set or test set (especially if the two come from different sources and/or have different distribution).

We dont have enough information to answer this question, especially as we don't know what problem you intend to solve, but here are the questions i would ask myself:

• is there a difference in distribution between train/test ? (eg. train is balanced and test is not, my intuition tells me it is the case)
• is there something I missed about the data? (no learner will ever be able to classify something as 8 or 11, if it has never seen any example of it !)
• is this a classification problem ? If saying class 1 < class 2 < class3 ... makes sense it is likely to be an ordinal regression. You can't "fill the gaps" (ie. predict an 8 without ever seeing one) with a classification, but you can do it with a regression (only if regression make sense).
• does this problem has a solution ? If you try to learn an impossible task, there will be overfitting, always and no way to prevent it.

Hope this will help