I have a dataset with approximately $70,000$ entries and $8$ features. Some of them are ordinal and the rest are nominal. The task is a binary classification task; however, the class I am interested in is represented only by $5\%$ in my dataset (highly imbalanced classes).
Some of the nominal features have many levels, so I have tried to group them (based on the relative frequency), such that I have at most 8 levels for every feature.
Moreover, I am interested in getting a fairly high precision in the minority class. I have tried
RandomForest, but in every case I get a precision of ~$8\%$.
I have implemented RandomForest, since there is no need to proceed to one-hot encoding and I know it generally performs well even in imbalanced data.
I don't know how else to proceed regarding this one. I was wondering if there is anything fundamental which I probably miss.
P.S. I can/will definitely try different classification algorithms, such as SVM (though I need to introduce one-hot encoding in my data).