I am trying to to a weighted implementation of Random Forests where i want to give greater weights to better performing trees in a forest.
I have currently split my data set into 60:20:20 (Train Set: Performance Measure Set: Test Set). I am operating on data sets where the minority class has an incident rate of less than 1%.
I am training my random forest on the train set, then measuring their performance on the Performance measure Set, assigning weights for each trees based on their performance and executing weighted prediction on the test set.
However results does not give me an improvement over the majority voting implementation
- What is theoretically wrong with my implementation? Should a weighted approach theoretically give better results? Wouldn't weighing of trees give better results for skewed data sets where trees trained on the minority class information should ideally be given more weights?
- How can i get a measure of tree performance from the training set itself? I have looked at the following field on the output from Rf - 'err.rate'. How can i get performance measures for individual trees from the training set?
- Can i rerun my model on the training set and look at votes from each trees to gauge each tree's performance.
Any help is highly appreciated.