let's say that we have a simple, binary classification problem (with many predictors and many observations) and want to fit for example some kind of boosting algorithm to obtain resutls. Let's also assume that we are able to identify two clusters with some kind of clustering algorithm. My concern is whether it is possible to obtain better prediction results when boosting algorithm is fitted separately for each cluster, not on the whole dataset? Theoretically, clustering is performed on the same variables as predictive model, but maybe there is some kind of logic in such a solution?
What is more, maybe it is worth trying different model for the other cluster and this might gain better results?