With the goal of improving out-of-sample performance on a general Random Forest Classification problem, what are other things one can do in addition to tuning a single RFC model's hyperparameters?
For instance, would it make sense or add any benefit to train multiple RFCs on different subsets of a training data set, then average the prediction probabilities of those separate RFCs to create a single prediction probability to apply to a given piece of out-of-sample input data? If that would be beneficial, would it be best to randomly select a subset of the training data set to train each RFC?
Or, would it be sufficient to do a random train/test split on a percentage of a training data set and train a single RFC on the split data, but do so X number of times, then select the RFC model that ends up with the highest overall accuracy on the test data to use for the final model? Would that technically result in an overfit model?