The answer is that yes, this is possible. This is addressed in the StackOverflow answer here: https://stackoverflow.com/a/28508619/6479831. The "model addition" is performed by modifying the estimators_
and n_estimators
attributes of the models.
The following code block is copied from the answer linked above, and demonstrates how to do this.
from sklearn.ensemble import RandomForestClassifier
from sklearn.cross_validation import train_test_split
from sklearn.datasets import load_iris
def generate_rf(X_train, y_train, X_test, y_test):
rf = RandomForestClassifier(n_estimators=5, min_samples_leaf=3)
rf.fit(X_train, y_train)
print "rf score ", rf.score(X_test, y_test)
return rf
def combine_rfs(rf_a, rf_b):
rf_a.estimators_ += rf_b.estimators_
rf_a.n_estimators = len(rf_a.estimators_)
return rf_a
iris = load_iris()
X, y = iris.data[:, [0,1,2]], iris.target
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=.33)
# in the line below, we create 10 random forest classifier models
rfs = [generate_rf(X_train, y_train, X_test, y_test) for i in xrange(10)]
# in this step below, we combine the list of random forest models into one giant model
rf_combined = reduce(combine_rfs, rfs)
# the combined model scores better than *most* of the component models
print "rf combined score", rf_combined.score(X_test, y_test)
However, see my comment on your original question for notes regarding whether this is advisable to do.
Edited to add details about what this does: A random forest is just a large ensemble of trees. Each of your K-folds produces such an ensemble. The code shown above simply combines all of these trees into one large ensemble. This is in principle not different from ordinary training of a full random forests model, since random forests train each tree on a bagged version of the data; K-fold CV essentially enforces constraints on the bagging that is used to build your trees. I suspect that in general you would see fairly modest improvements from a model trained on the full dataset in comparison with a model built from combining ensembles built under K-folds (all else equal).