# Model averaging in prediction — “Wisdom of the Crowd”

Suppose I'm trying to predict $Y$ (a real number) and I have $n$ experts with guesses $Y_1,...Y_n$. Each prediction is a reasonable guess as to the value of Y in itself (hence the name "expert"), but we should be able to come up with a better prediction for $Y$ by combining the experts' guesses. We could take our guess $Y_o$ to be

$$Y_o=\sum_{i=1}^n\frac{Y_i}{n}$$

but this isn't optimal if some experts are better than others.

I have a few questions related to this topic.

Question 1: Is this equal weights method used much? Are there any other simple methods? Can you refer me to (preferably relatively basic) information on the topic?

Question 2: I've seen model averaging discussed in this as a possible alternative to other methods of model selection for linear regression which may fail, for example, if there is multicollinearity. That's led me to wonder whether the following solution might be justifiable:

We want (perhaps out of naïveté) to regress $Y$ against predictors $X_1$, $X_2$ and $W$. Suppose, in fact, that $X_1$ and $X_2$ themselves are predictors of $Y$. Using multilinear regression for $Y$ with all three variables is problematic because, being predictors of $Y$, $X_1$ and $X_2$ exhibit multicollinearity, but what about creating two multilinear regression models

$$Y=\alpha_o + \alpha_1X_1 + \alpha_2W$$

and

$$Y = \beta_o + \beta_1X_2 + \beta_2W$$

and then using some average-like combination of these two regression-based predictors to predict $Y$? What weights might we use? Again, is there any literature on this topic?

Thank you.

EDIT: Since I posted this, I've read about stacked regression, as described here. Does the situation I've described sound suitable for applying stacked regression?

-
Note that multicolinearity is not a problem for predictive purposes as it just means there is more than one optimal model. you can write the least squares estimate of $\beta$ as $\hat{\beta}=X^{+}Y$ where $X^{+}$ is the moore penrose pseudo inverse of $X$. this reduces to the usual least squares estimate when $X$ has full column rank but still exists uniquely in case of multicolinearity. it resolves the non uniqueness of the original least squares estimates by taking the beta with the smallest sum of squares and corresponds to limiting ridge regression as penalty goes to zero. –  probabilityislogic Mar 23 '12 at 23:04