How to get rid of bias in data? I have been trying to classify a set of data into one of four classes. The data has already been generated and I have set aside 10,000 for training and 2,000 for testing. I have also generated the labels for each of the data. Let's call the classes - 0,1,2 and 3.
Now when I observe the classification, I notice that there are a lot of 0s in the training data and hence in most cases, the classifier is just learning to predict 0 no matter what the features are. (I am using random forests for classification)
Generating the data again to ensure uniformity, takes a lot of time and I prefer to avoid that. Is there anyway I can still use the data that I have?
 A: Another way is to oversample: "Oversampling: you duplicate the observations of the minority class to obtain a balanced dataset." [1]
But note that oversampling of the minority class may lead to overfitting, so be sure to test that. 
You also may want to check this paper: Yap, Bee Wah, et al. "An Application of Oversampling, Undersampling, Bagging and Boosting in Handling Imbalanced Datasets." [2]
A: This is usually referred to as class imbalance or skewed data not bias.
For a random forest you can use roughly balanced bagging to resample the data used to grow each tree during the bagging process.
You can also look into using a weighted or cost sensitive criteria for tree growth like weighted gini or entropy. Note that weights should be tuned using a grid search or hyperparameter optimization as it is difficult to guess good ones. IE weighting the majority class more then the minority class may produced the best balanced error somewhat counterintuitively.
Finally heilinger distance decision trees have recently been proposed as less sensitive to this sort of things.
I wrote a random forest implementation that includes a bunch of different methods for imbalanced data. 
