Is model selection just a specific kind of ensemble learning, where ensemble learning is loosely defined as "combining multiple models in some capacity to hopefully get an improved model"?
It would seem that model selection does indeed fit that definition. My thinking is that the model selection process considers a set of models and at every point in the sample space chooses the model from the model set which optimizes some criterion at that point. The result is thus a new model, defined piece-wise across the sample space according to the selection criterion.
Is this correct, or am I missing something here? I can't seem to find a definitive answer to this question in the literature, hence my post, but the paper "Estimation and Accuracy after Model Selection" by Bradley Efron seems to support this viewpoint. I've also heard of ensemble learning via a "bucket of models" (a description of which can be found on the "Ensemble Learning" wikipedia page) which seems related to what I am asking about, but I could just be misunderstanding that too.
Edit: Upon further reflection, it seems like frank's answer answers my question. By thinking of estimation as a specific kind of prediction problem (in the sense of using data to predict\guess what the parameter value is), it seems like model selection can be thought of as a specific kind of mixture of experts, which in turn is a specific kind of ensemble learning. Thus, model selection is a kind of ensemble learning.