Combining models for prediction based on residual performance

I have never read or seen someone do this before, so I wanted to pose the question here. Suppose I fit a basic linear model, $\text{price of house} = \beta_0 + \beta_1*\text{taxes} + \beta_2*\text{bedrooms} + \beta_3*\text{number of garage doors}$.

I fit this using OLS and I get a model that has combined $R^2$ of say .68... and I look at the residuals from the actuals vs predicted plot, and I see that the model does pretty well for houses that have less than 3 bedrooms, and maybe more than 2 garage doors....

I then build a robust linear regression model, or maybe a neural net, and I get a model that now performs worse as a whole, with a combined $R^2$ of say .53 BUT when I look at the residuals from the actuals vs predicted plot, I see that this model, model #2 was actually better a predicting the sales price for houses that model #1 did a poor job of predicting, say the outliers.... this scenario goes on and on, and lets say I have identified 5 different models, that for different parts of the population, do a better job of predicting those parts of the populations and these parts are not mutually exclusive.

Is there are known approaches or good documentation on say using diagnostics to identify where your model may be poorly performing (on what types of data), and then some standardized approach to say joining predictions from multiple models to get the highest $R^2$? I am not suggesting averaging model predictions, or what is generally seen in ensembles, at least in how I have seen them defined. I am saying a tactic to apply different types of models to different parts of a dataset, (through semi-automatic identifcation of which model is best applied where), and then joining up all of the best predictions.

I do not think segmenting the population first, and then building seperate models for each segment of the population is the best answer here, but is certainly one way to pseudo tackle it.

Hopefully this concept makes sense, as I think it would be very powerful. Thanks to everyone in advance for providing your thoughts and or supporting documentation / literature.

I've used a competitive mixture of experts ensemble successfully in the past to solve this type of problem. In this formulation, the key is that you are training the partitioning/gating function (typically the softmax function) and the underlying experts (linear regressions) at the same time. In this manner, the input data windows that are passed to a particular expert have input attributes that are in similar portions of attribute space but also share a similar relationship to the truth output. Typical mixture of experts papers employ a cooperative framework, in which the output is the sum of the regression output of each expert multiplied by the probability that the expert is responsible for the window of data (with the probability determined by the output of the softmax gating network). In a competitive framework, it is just winner take all; the output is just the regression output of the expert that had the highest probability.

Relevant papers: