splitting a training data set based on classifier accuracy I have training data with ~600K instances. If I split the training data into four segments and build four separate classifiers, I get much higher accuracy for each model than if I train a single model over the entire data set. 
I can do the split for this specific data set because I have prior knowledge of the data. I am wondering if there is an algorithm that will automatically do this task for me. 
Given a labelled training data set, divide it into k subsets for k classifiers that are likely to result in higher accuracy. 
So something like random forest, but without the "randomness"
 A: One way of approaching this issue is to try ensemble methods such as bagging, boosting. These approaches generally perform better than single model. This paper would be a good starting point.
A: The apparent performance of a method will go up as you decrease the sample size, because the signal:noise ratio (amount of overfitting) gets worse.  But you have an additional problem in using an improper accuracy scoring rule, which is akin to adding a lot of meaningless noise to the accuracy score no matter how large the sample is.  Using an improper accuracy score for judging the relative merits of different models will lead to incorrect judgements a significant portion of the time.
A: If you are looking to automate the clustering, you could use a k-means or hierarchical clustering approach.
If you want an algorithm that would take your clusters into account, a non-linear approach such as boosting or random forest would probably work well. If the clusters you've identified are important, the trees should split along those features and then you don't have to manually specify the clusters.
