Model uncertainty (model averaging) and R-Squared ($R^2$) Is it possible to calculate r-squared for an "average model"?
Lets say I have 4 different response variables that I want to model to a set (or subset) of 4 independent variables.  I'd then like to compare the variance explained (R-squared) for the best model for each response variable.  Unfortunately, for each of the 4 response variables there is no clear best model, so I need to model average to account for this model selection uncertainty.  Now, is there still a way to compare the variance explained by the averaged model for each of the response variables?  Or do I simply compare the r-squared values of the global model for each response variable?
My question is similar to this old unanswered question;
Generalized $R^2$ for average model
Thanks for any help and please let me know if I can improve this question.
 A: The following book addresses model selection and averaging, including frequentist and Bayesian approaches, use of AIC and BIC (an alternative to R2 in your case?) and more: http://www.cambridge.org/us/academic/subjects/statistics-probability/statistical-theory-and-methods/model-selection-and-model-averaging.
I may be wrong, but I assume that your question, possibly indirectly, is related to ensemble methods (http://en.wikipedia.org/wiki/Ensemble_learning) and associated topics and methods. I was curious about your question and did some brief research. I'm not sure whether Bayesian approach can be applied to your models, but I hope that the following resources will be helpful for you in regard to your question.
Combining models, Bayesian:


*

*A tutorial on Bayesian model averaging (BMA): http://www.stat.washington.edu/www/research/online/hoeting1999.pdf

*Ensemble BMA and forecasting: http://andrewgelman.com/wp-content/uploads/2014/03/EBMA_conditions6.pdf

*Bayesian model combination instead of BMA: http://axon.cs.byu.edu/papers/Kristine.ijcnn2011.pdf
Combining/comparing models, non-Bayesian:


*

*The 2008 BellKor Solution to the Netflix Prize: http://www2.research.att.com/~volinsky/netflix/Bellkor2008.pdf

*Paper on average predictive comparisons (APC) of models with non-linearity or interactions: http://www.stat.columbia.edu/~gelman/research/published/ape17.pdf

*Blog post on APC method R implementation: http://andrewgelman.com/2014/06/17/average-predictive-comparisons-r-david-chudzicki-writes-package

*Corresponding R package: http://www.davidchudzicki.com/predcomps
A: I am not entirely sure how. I think you could work it through. Basically, propagation of error is the total derivative of your fit equation, with covariances, example. When you do your fit routine, with what ever method is appropriate to your problem, just get make sure to use one that gives you the standard deviations of all of the parameters. Then you can solve for any combination of parameters' variances that you want to. I did that adaptively in the Appendix section of this paper. That is, I not only solved the error propagation for my parameter of interest, but I optimized for it.
For your problem, for the total error of each of the dependent variables, you have a contribution from each of the error propagation terms. Now the added confusion of using a "population model," depends on exactly how you obtain it, and after that I am lost in what you are doing, exactly. However, I do not think that it changes the rules for error propagation, just what the errors themselves are.
I am going to take a guess as to what an average model is, and mention that if that means that we fit models separately, then average the results, that one might start with a mixture model, and fit that, as following that one might get a better fit, and, then calculation of $R^2$ is routine. For example, $s (p b_1 e^{-b_1 t} +(1-p) b_2 e^{-b_2 t})$ is a mixture model scaled by $s$ times a unit CDF($\infty$) consisting of the sum of two independent exponential distributions, that is one of the most commonly used pharamcokinetic models. If you are doing autocorrelation or some other averaging, please let me know and I will change my suggestion to match it, if I can.
Another possibility is to noise reduce the data using adaptive B-splines or locally adaptive Pixon. What should be done depends on what the purpose of obtaining an average is.
