Multi task learning I have a dataset where all observations are measured several times and reported outcomes correspond to those measurements. In other words, my set of data points looks like
$\{x_i, y_{i_1}, y_{i_2}, ... y_{i_n}\}$
I would like to make a predictive model to predict $y$ from $x$. What should I try to predict? The average of $y_i$? 
I've heard something about multi task learning, where each observation is considered as $n$ different regression problems and all problems share the same kernel. Can anybody elaborate on this?
 A: If observations (for the same subject) are uncorrelated, then one thing you can do is to average observations and use some prediction technique (e.g., regression). However, if these observations are correlated, then the data set is longitudinal.. then you need to use, say generalized linear mixed models...
A: If the observations are independent, and the ordering of the observations is unimportant, then you could simply estimate the mean as Actuary suggests (+1); however the fact that multiple observations are made suggests that the spread of the observations may be of intrinsic interest, in which case it would be a good idea to model the distribution of plausible values for the observation (for a Gaussian, this would be modelling the conditional variance as well as the conditional mean).  
I did some work on predictive uncertainty a few years ago, and the method described in 
G. C. Cawley, N. L. C. Talbot, R. J. Foxall, S. R. Dorling and D. P. Mandic, Heteroscedastic kernel ridge regression, Neurocomputing, vol. 57, pp 105-124, March 2004. 
might be of some interest (MATLAB demo here).  It ought to be fairly straightforward to make a basic multiple instance version assuming the observations are conditionally i.i.d.
BTW I think this kind of problem is known as "multiple instance learning" in machine learning, there is a fair amount of research on this indexed by Google Scholar, so hopefully there may be something relevant to your problem there somewhere (too many papers, too little time to read them... ;o).
A: All of the answers above suggest techniques that assume the data are uncorrelated (with the exception of the later part of user9292's response).  I highly doubt that is the case here since you have multiple observations from the same subjects.  As a results, you'll likely to take the correlation into account using methods such a mixed models, generalized estimating equations (GEE), or transitional models.  This site has plenty of information on all of these methods and how you might be able to apply them.
A: Basically you can use Pareto Front  when choosing between different solutions. One particular approach which extensively uses Pareto front is genetic programming. There are several good frameworks for this stuff where you can setup you problem and see the result without months of coding, check answer to this question with list of frameworks. 
