I have a training data which after cleaning, wrangling has around 17k with more than 40 columns. The categorical columns are 35 and numerical 5. The categorical columns have value either 0 or 1. Now the thing is this training set is highly imbalanced as for every column I have, it has roughly 90% of the column as 0 and 10% like 1. I'm trying to predict prices using these columns. As I try to predict values using these columns I get a training score of 80% and validation of around 43-44 %. I have used Random Forest, XGBoost regressors for prediction. I have also tried cross-validation, hyperparameter tuning for these but the results don't seem to improve. Is it that the data is not right - being highly imbalanced and all or is it a problem with the model . How do I improve this crappy model ?
P.S : I can't share the data or code due to company policy