How to fit to sum of observations? From a practical point of view, how does one go about fitting a model to training data that consists of sums of a dependent variable over multiple conditions? For example, fitting a model to predict the incomes of individuals given only the incomes of households and the descriptors of each member of each household.
To be more precise: I wish to predict a dependent variable $y$ in from $n$ independent variables described by a vector $\vec{x}$. My training data does not consist of the usual sort of observations $(\vec{x}_i,y_i)$. Rather, I have the sums of various mutually exclusive subsets of ${y_i}$. In other words, my training data consists of $\{\vec{x}_i\}$ and $\{Y_k\}$, where
$$Y_k=\sum_{i\in{J_k}}{y_i}$$
The values of $\{J_k\}$ are known and are mutually exclusive, meaning that each $i$ is contained in one and only one $J_k$.
I would like to train a model $f(\vec{x})$ with this data using off-the-shelf tools. For example, using scikit-learn to fit a random forest regression. It's not clear to me how to do this through the API, which seems to require the training data to contain observations $(\vec{x}_i,y_i)$.
Also, what is the best terminology to describe this sort of optimization problem? Is there a specific name for it?
 A: As you have noted in your comment, if your model is $y_i = w' x_i + e_i$, it implies that $Y_i = \sum_i y_i = w' \sum_i x_i + \sum_i e_i$ and performing a least squares fit of $Y_i$ from the sum of the input groups $\sum x_i$ would work well.
This can be generalized in a straightforward way to non-linear models where $y_i = w' \phi(x_i) + e_i$, where you can view $\phi(x_i)$ is a high-dimensional (kernel) expansion of $x_i$. Then $Y_i = w_i \sum_i \phi(x_i) + \sum_i e_i$. 
Now if $K(x_i, x_j) = \phi(x_i)' \phi(x_j)$ is a given kernel function over your input space, the kernel function $K_s(\sum_i \phi(x_i), \sum_i \phi(x_j) = \sum_i \sum_j K(x_i, x_j)$. 
Therefore a least squares solution using a given p.s.d. kernel function over your input space, can be done using your aggregate samples by constructing the sum kernel $K_s$ over your example sets $\{x_i\}$ and building a kernel regressor (least-squares SVM, say) to $Y_i$ and then using the solution weight vector with the original kernel $K$ to predict $y_i$.
This is all speculative, and I have no guarantees for how well it works for your problem.
A: The first thing that comes to mind is an iterative approach. 
In Step 1, you assume that each y_i contributes to Y_k equally, and do the fit. You'll get a way of predicting y using x. 
Now in Step 2, you use your predictions for y_i from Step 1, normalized so they still add up to Y_k. You'll get a different model to predict y using x.
Repeat the steps until your predictions converge/plateau.
I think you can set this up with most standard tools.
