Batch Learning w/Random Forest Sklearn I have a data set of approximately 5 million rows and wanted to run a RandomForestClassifier. I ran my RandomForestClassifier with only 50 trees, I tried to use the fit function but I receive a memory error. I tried running it on the AWS box with 64GB worth of memory but still I run into this issue. 
I was wondering if it was possible to use some sort of Batch Learning to overcome this issue using sklearn? I am open to other suggestions if anyone has any.  
 A: Yes, Batch Learning is certainly possible in scikit-learn. When you first initialize your RandomForestClassifier object you'll want to set the warm_start parameter to True. This means that successive calls to model.fit will not fit entirely new models, but add successive trees. 
Here's some pseudo-code to get you started. This will build one tree for each sub chunk of your data. 
# split your data into an iterable of (X,y) pairs
# size each one so that it can fit into memory
data_splits = ... 

clf = RandomForestClassifier(warm_start = True, n_estimators = 1)
for _ in range(10): # 10 passes through the data
    for X, y in data_splits: 
        clf.fit(X,y)
        clf.n_estimators += 1 # increment by one so next  will add 1 tree

I'm surprised to see that there's not a subsample parameter in RandomForestClassifier similar to the one in GradientBoostingClassifier that controls the number of observations visible to each tree. If you switched to GradientBoostingClassifier you might be able to simply set subsample to be a very small number to achieve the same results. 
