I have been working on an ML model, the data is about user customer desk tickets. To start easy, I decided to classify (if possible) users who are likely to raise support tickets. My positive class is just a little over 3.5% and I have tried using ensembling, under and oversampling and the results aren't really pretty. I took a PCA transformation on more than 800 features as they were one-hot encoded. Built a model on both (pre and post PCA transformation). And now finally( which I think I should have done earlier), plotted a 3d plot of transformed vectors. Visually, I don't see a pattern here, yellow being a positive class. Could it be a correct way of looking at the problem and deciding that the occurrence of such tickets are random in nature or more robust feature engineer may help?
All features are normalized and I have tried Oversampling(SMOTE, kmeans) and undersampling with near miss so far.
Results with hyperparameter tuning on random forest, GBoost and even neural net are almost similar. Below are my result and scatter plots.