My aim is to predict a binary output, given instances X and associated binary outputs y. I built a first classifier out-putting scores (not calibrated in probabilities). However, I identified in my dataset two subpopulations (different data generating processes, poor performance of the model on one of them). So I splitted my model in two, and refined my new models independently. The main refinement has been adding specific variables to both subpopulations.
Would it make sense to merge the scores given by my models ? Do models need to be calibrated in probabilities before being merged or could calibration in probabilities be achieved after ?