multiclass classification when x variable is not well spread and class imbalance

Question

I have a dataset as below. It is a classification problem with multiple x variables and a y variable. Y variable has 5 levels. The below picture has y variable on x axis and one x variable on y axis. As you can see from the picture below, for each value of Y, a x variable distribution is not very different (i plotted similar chart for all x variables- one x variable at a time vs y and seeing the same trend). Due to this issue, most of the observations are getting classified as class 0 or 4. I built a randomforest with 500 trees. How could i improve accuracy? I thought of taking square or cube of each x so that distance between class means would increase but it won't help with variance of each class

distribution of 5 classes in my data is as below

   0    1    2    3    4 
5104 2639 2322 2661 5274 

Column means and standard deviations of my x variables are very similar :(

---

update 1

I have 26 x variables and 5 outputs. As mentioned above most of the datapoints are being predicted to class 0 or 5. I performed one additional model where for each Y class i found out mean of each x variable. That step provided me 26*5 means. Then for each observations and each x variable i found out squared distance between the datapoint and class centers and summed it for each observation. This would return me 5 distances for each observations and I assigned that observation to a class where distance is minimum. This approach is much better.

The approach in short:

1. find mean of x variable for each Y class - for example i got x1_mean_for_y=0
2. find squared distances for each value of x column - for example squared distance between (1st observation of x1 and x1_mean_for_y)
3. for each datapoint some all distances by class
4. assign a datapoint to a class where the distance is minimum

This approach assigns relatively less number of observations to class 0 and 4 and my overall accuracy has gone by 7%

how could i improve my model further?

Answer 1

Transforming individual variables would not help in random forest, because you can transform distances, but order of observations would remain the same, and problem of inability to distinguish classes would remain. In fact in most decision trees distances does not matter at all and only order of observations matter.

Unfortunately, if for some groups of cases your data does not differ between different classes, it is impossible to predict them correctly.

However, there is a hope. Although, your classes does not differ on any single variable, there is a chance, that they differ in multidimensional space of those variables. Try using some algorithms that are multidimensional (does not use single variable splits like decision trees in random forest, but many variables at once) or uses distances. I would recommend trying: -k-means (this of course uses distances on multidimensional space, you should try with transforming variables with PCA before modelling and experiment with 5+ centroids), -multinominal logistic regression with quadratic terms on every variable (this method gives score basing on many variables at once, so maybe it would help), -support vector machine (quite complicated to explain it here, but it also uses kind of multidimensional distances).

If your problem is only the imbalance of predicted classes you can retrieve votes of every tree from your model and decide yourself about some custom rules what class should be predicted with certain votes share or you can weight votes for different classes. Some random forest implementations also allow to assign weights to model. If you would weight underrepresented classes with higher weights your model will predict them more often.