Machine Learning Algorithms for Predicting Entire Regression Functions From what I've seen in (supervised) machine learning the general idea is to work with some training set $(\mathbb{x_i},y_i)$ and learn the $y_i$ outputs without over-training. In the case of regression this is a continuous problem. However, in the previous definition of my training examples (and in a lot of definitions I see) the $y_i$ is single valued. So I was wondering if it were possible / if there are any good machine learning algorithms currently which can work on continuous vector value output spaces?
To put it more simply, imagine I have a set of inputs, which I regress to a particular output. We can call this model $H_1$. I have new set of inputs, and a new set of outputs, which results in a new model, $H_2$, and so on until I make up to $H_n$ models. I want this set of regression functions to by my training data set, so that if I have new input, I will be able to make an educated guess on what this new model, $H_{n+1}$ should be. 
I can only think of possibly extending Boosting to this scenario (if it hasn't already been done before), perhaps using a Gaussian Process, or maybe even some Bayesian parametric method where I integrate over all the models? I don't have much experience using entire regression functions as training inputs, so I'm wondering if someone on here could educate me on the topic / suggest plenty of algorithms and point in the direction of some good sources.
 A: Under the assumption that you're looking at similar data across the different regressions, you're describing Bayesian hierarchical modeling. The classic example of this is the Eight Schools data: students from each school are pre-tested in PSAT scores. Then each of the students' schools provide PSAT coaching to them. Then the students take the PSAT again. Was one coaching experience more effective than the others? How much do the students' scores improve (if at all)? A hierarchical analysis of the PSAT scores can answer this question.
Using the posterior predictive density from the model and some pre-coaching data for a hypothetical ninth school, we can make inferences about how effective coaching might be for this new batch of students.
Hierarchical models are premised on the idea that regression coefficients themselves have some distribution, e.g. $\beta\sim\mathcal{N}(\mu_\beta,\sigma^2_\beta)$ would imply that the coefficients $\beta$ are  normal deviates with a mean and variance parameter.
Gelman's Bayesian Data Analysis is a great resource for getting started with hierarchical models.
