Do model averaging and model combination mean the same? I am not sure, but I guess model averaging and model combination and even forecast averaging and forecast combination are used arbitrarily in the literature...
Is this only my feeling or indeed the case? Any reference?
 A: There are different ways how we can combine models (e.g. majority vote in classification) and averaging them is only one of the approaches.
They are not that arbitrary as for particular algorithms, for example as described in Wikipedia article on ensemble learning,

Bayesian parameter averaging (BPA) is an ensemble technique that seeks to approximate the Bayes Optimal Classifier by sampling
  hypotheses from the hypothesis space, and combining them using Bayes'
  law. [...]
Bayesian model combination (BMC) is an algorithmic correction to Bayesian model averaging (BMA). Instead of sampling each model in the
  ensemble individually, it samples from the space of possible ensembles
  [...]

Notice also that there is also a difference between averaging (or combining) the models and the forecasts (i.e. outcomes of the models). In first case you average the models themselves and in the second case you use each of the models to make individual forecast and then you average the forecasts.
Nevertheless, I agree that sometimes all the terms are used exchangeably, what may be confusing.
A: On a theoretical level, Bishop in $\S$14.1 and Murphy in $\S$16.6.3 insist on distinguishing between Bayesian Model Averaging and Ensemble Learning/Model Combination.
Namely, in Bayesian Model Averaging we account for our uncertainty in the true population model $$p(y | x) = \sum_k p(\text{model} \ k \ \text{is the true population model}) p(y |x, \text{model} \ k \ \text{is the true model}),$$and in principle if the true population model is one of the $K$ models we are considering, say $k=1$, with enough data the posterior $$p(\text{model} \ 1 \ \text{is the true population model} \ | \ \text{data}) \to 1$$ will select the true model, and we will be using $$p(y|x) = p(y|x, \text{model} \ 1)$$ for prediction.
In Ensemble Learning we on the other hand postulate that the true model happens to be a mixture of other simpler models, e.g. the regression function can be postulated to be $$f(x)=E_y(y|x) = \sum_k w_k f_{\text{model} \ k}(x).$$
Even after more and more data is available for estimating the weights $w_k$, the estimates don't select one model (like e.g. $w_1 = 1, \quad w_i = 0 \quad i \neq 1$) as in Bayesian Model Averaging, reflecting the composite nature of the regression function. Furthermore, the weights themselves can depend on the input point $w_k(x)$, allowing for even more flexible mixtures.
On the more practical level, since not only the population model is unknown, even its functional specification can rarely be guessed right. That is why I think in ESL the authors are not pedantic about the above distinctions and instead discuss various practical ways of combining models like bagging ($\S$8.7), committee methods ($\S$8.8), boosting ($\S$10), etc.
