Minimizing Curve fit for predictive model Let's assume we've found 100 independent variables that can predict y. Each of those independent variable are close to uncorrelated and they are all curve fitted. Using any single one to predict y equates to a large probability of failed out of sample predictive performance. But if we combine all of them via some sort of ensemble averaging, the final prediction would be more robust. The logic here is similar to diversifying across each independent variables predictiveness since it's highly unlikely that all of them will fail together. Does this logic make sense? Are there any literature in predictive modelling that discuss this subject more? 
 A: In addition to my comments above and links within, I'd like to share a bit more information, which is hopefully relevant and helpful. It seems that ensemble methods, such as ensemble Bayesian model averaging (EBMA), indeed improve predictive ability of individual models as well as offer certain other benefits. For example, Montgomery and Hollenbach (2012) write:

Yet, combining forecasts, and ensemble methods in particular, have
  been shown to substantially reduce prediction error in two important
  ways. First, across subject domains, ensemble predictions are usually
  more accurate than any individual component model. Second, they are
  significantly less likely to make dramatically incorrect predictions
  (Bates and Granger 1969; Armstrong 2001; Raftery et al. 2005).
  Combining forecasts not only reduces reliance on single data sources
  and methodologies (which lowers the likelihood of dramatic errors),
  but also allows for the incorporation of more information than any one
  model is likely to include in isolation.

A paper by Singh, Mishra and Ruskauf (2010) provides an interesting comparison of a subset of model averaging techniques, which include three types: frequentist, Bayesian and information theory-based. Finishing on a practical note, I would like to share a page from a popular Python machine learning library scikit-learn, dedicated to several frequentist ensemble methods.
References
Montgomery, J. M., & Hollenbach, F. (2012). Improving predictions using ensemble Bayesian model averaging. [Working paper] Retrieved from http://pages.wustl.edu/montgomery/ebma
Singh, A., Mishra, S., & Ruskauff, G. (2010). Model averaging techniques for quantifying
conceptual model uncertainty. Ground Water, 48(5), 701-715.
