How to choose classification model when number of observations in one class dominate the other? My dataset is about sales in a department store, I have sales from previous years as my predictors, and I want to predict if a customer will or will not purchase this year.
I used random forest first, and here's the result of my confusion matrix:

The model is good at predicting 0s, but did a poor job predicting 1s. 
My guess is that this is because we have 25 times as many 0s as 1s in the response variables, thus, even if the model predict 0 for all observations, the error rate would still less than 4%.
I also tried taking out variables that might lead to multicollineary. And because my predictors are highly right-skewed, I also tried to use log/sqrt/reciprocal transformation, but none of those works.
Thus, I am wondering what kind of model, or what is the general approach when the number of one class outrun the number of the other class in response variables?
Any idea/comment/suggestion is appreciated
 A: If you think your empirical prior (class distribution of training set) is unreasonable during future prediction you can adopt a different prior e.g. a uniform prior (equal intrinsic probability of either class).  Yes, you can use classwt(weighting samples by class) but strata(downsampling prevalent class) yields same performance and is better to control the effect of:
rf = randomForest(...,#all your other pars inserted as usually
 strata=train.np.log$targamnt #insert your training targets(as factor here)
 sampsize = c(344,344) #down sampling for each tree bootstrap
 replace = TRUE #if set to FALSE sampsize should be c(211,211)
)



*

*don't bother colliniarity, as RF have no problems with that. Perhaps try dropping a fraction of the features with lowest permutation importance.

*don't bother skewed features. Decision tree splits are non-parametric and only mind the ranking(ordinal scale). All your transformations are monotonic, thus the ranking of feature values are unchanged and you will get the exact same model result (when using set.seed(123))

*Use a better performance metric than class.error and confusion matrix such ROC plots

*Decide what trade-of between sensitivity and specificity works the best for your case


here's a link to full code solution in R with randomForest and explanation on cross validated
You don't need to switch to adaboost, as it is likely inferrior to RF. RF may be inferrior to xgboost, gbm or svm. But basicly, you can handle skewed data with all packages and models.
A: I encountered a problem once where a data set was highly skewed towards one class than the others.  To accommodate for this, I built a training set that contained a more balanced number of cases for each class.  This generally resulted in a significantly greater classification rate for the smaller class and a minor decrease in performance with the larger class.
There are other models or techniques that may better assist with the issue of skewed data, but I would recommend this as a starting point to determine if it has any affect on the quality of prediction for your problem.
A: 
or what is the general approach when the number of one class outrun the number of the other class in response variables?

In general, most classifiers work by minimizing a cost function which can be adapted to take into account the imbalance. 
For instance, in SVMs the empirical risk to be includes a term L(y(x), t), which stands for the cost of classifying x as y(x) when the true label is t. If we have a problem where misclassifying A as B is different from misclassifying B into A (which is sometimes the case in imbalanced data), then we can specify those costs in L. (It's just we sometimes forget that this is possible because, by default, SVMs and other classifiers employ the 0-1 loss which gives the same penalty for misclassifying classes). 
In the case of SVMs, logistic regression and many other classifiers, this imbalanced loss function can be handled equivalently by re-weighting the data (i.e. using the "default" loss function but assigning weights to the different samples). Some known implementations of such algorithms allow specifying those weights as input. 
Now quick feedforward to random forests, according to this in R you can use the classwt parameter to set class weights in a random forest classifier. 
A: There are few ways u can improve the model : 


*

*Boost the data for the class which has less data samples.

*You should also look for adaboost in R . This algo. boosts the class in the dataset, for which your model is not performing good and again trains on that ,and does this again and again. 

*Instead of taking 0.5 as threshold,u can change choose the appropirate threshold by plotting AUC-ROC curve. 


I think you must give try to adaboost as this is apt for your case.
R package : adabag
