Given two partially overlapping datasets $X_1$ and $X_2$ (say past 10K hours and past 10K minutes), how could one go about creating an ensemble model of classifiers of these datasets? Standard approaches such as boosting or stacking will not work, as the datasets are only overlapped on the most recent 1.6%. Majority voting seems very basic/inaccurate.
I was thinking of implementing some feature space accuracy function for the predictors (i.e given a point on a feature space, how well do you expect the predictor to perform, based on previous predictions around that point) and then take accuracy weighted votes using the above. Has this been done before? Is this a good idea?
What's the best way to approach this? Any techniques/pointers to literature are appreciated.