I am using a regression approach to make predictions using a variety of variables. However, some of my variables are pretty collinear (with a Pearson's r > 0.75), so I can't include them all in the regression. Furthermore, I am finding that the model predictions are pretty sensitive to which variables I include as predictors. One approach I am considering is to use an ensemble of regression models, where each model is based on a different combination of predictor variables so that no two are collinear. My thinking is that this might even help me estimate upper and lower limits of my predictions. However, I can't find any examples of studies that have done this (from any field), so perhaps there is a reason why one would not want to take such an approach?
Some background on why I am trying to do: I have a dataset of (continuous) child nutrition outcomes from multiple countries, and the outcomes were observed across a wide range of geographic conditions as well as meteorological conditions (some children were observed during drought years, others during normal years). I am trying to make a model of child nutrition outcomes under drought based on a variety of geographic factors, such as rainfall levels, GDP, national imports, and irrigation, and then use the model to predict how drought would affect child nutrition based on all of those factors. I am using regression because I am interested in having an interaction term between each geographic variable and a categorical variable for whether or not drought is occurring, and regression seems to be the best way to model this specific interaction. Furthermore, I am controlling for a variety of other variables that aren't geographic, such as the child's birth order, which also lends itself to a regression approach.
What I'd really like to know is: Has this sort of approach been done before? If so, where can I find literature on how to do it right or at least examples of studies that took this approach? If not, why not? Are there certain reasons why one would not want to use an ensemble of regression predictions?