Suppose I'm trying to predict $Y$ (a real number) and I have $n$ experts with guesses $Y_1,...Y_n$. Each prediction is a reasonable guess as to the value of Y in itself (hence the name "expert"), but we should be able to come up with a better prediction for $Y$ by combining the experts' guesses. We could take our guess $Y_o$ to be
but this isn't optimal if some experts are better than others.
I have a few questions related to this topic:
- Is this equal weights method used much? Are there any other simple methods? Can you refer me to (preferably relatively basic) information on the topic?
- I've seen model averaging discussed in this as a possible alternative to other methods of model selection for linear regression which may fail, for example, if there is multicollinearity. That's led me to wonder whether the following solution might be justifiable:
We want (perhaps out of naïveté) to regress $Y$ against predictors $X_1$, $X_2$ and $W$. Suppose, in fact, that $X_1$ and $X_2$ themselves are predictors of $Y$. Using multilinear regression for $Y$ with all three variables is problematic because, being predictors of $Y$, $X_1$ and $X_2$ exhibit multicollinearity, but what about creating two multilinear regression models
$$Y=\alpha_o + \alpha_1X_1 + \alpha_2W$$
$$Y = \beta_o + \beta_1X_2 + \beta_2W$$
and then using some average-like combination of these two regression-based predictors to predict $Y$? What weights might we use? Again, is there any literature on this topic?
EDIT: Since I posted this, I've read about stacked regression, as described here. Does the situation I've described sound suitable for applying stacked regression?