Using partial measurements of output variable in modeling My question is: How can we use partially measured output data in a training set?
This is vague, so I concretize it in a whimsical tale.
Squirrels Have Nuts, But How Many?
Setup
There is a set $S$ of squirrels and a set $T$ of trees in the forest. Squirrel $s_i \in S$  has $n_i \in N$ nuts. Squirrels store their nuts in one tree or many different trees in the forest. We wish to predict how many nuts a given squirrel has collected from input features: squirrel weight, and cheek capacity.
Research phase 1:
We monitored a subset of squirrels. Took their measurements (weight and cheek capacity), and counted

*

*how many nuts they collected and

*how many trees they stored them in

(e.g. Squirrel #55 stored 5 nuts in one tree, 10 nuts in another, and 500 nuts in another [which 3 trees is unknown])
Research phase 2:
We monitored a subset of trees. When a squirrel arrived at one of our trees, we I.D.'ed them (to know if they went to another tree in our study subset later), measured them (weight and cheek capacity), and counted how many nuts they dropped off. This gave us a partial nut collection measurement for a subset of squirrels.
(e.g. in Tree #23 we collected 10 nuts from Squirrel #99 and 50 from Squirrel #88, in Tree #24 we collected...)
(important note: Squirrel IDs don't persist across phases.)
Question
Suppose we wish to model the number of total nuts of a squirrel from phase 2. How could we use their partial nut measurements to augment the modelling results?
Furthermore, how can we introduce partially measured outputs into the training set?
 A: In phase 1 you could make a model that relates the behavior of a squirrel to the total number of nuts. 
In phase 2 you do not observe the exact same full information as in phase 1. For instance, you do not know how many trees in total are used by a specific squirrel. But you do observe some of the squirel's behavior, namely a sample from the distribution of the number of nuts per tree. From this you can estimate the distribution and the parameters that describe the distribution can be input for the model.


*

*So in phase 1 you make a model that relates the total number of nuts that a squirrel stores with the distribution of the nuts per tree for the squirrel. How exactly to model this is difficult to say. 
If you think that you could make some mechanistic model then you could start with some exploratory analysis and prior insights about squirrel behavior to get an idea about a useful model. I lack the data and the biological knowledge to do this in this answer (One obvious direction might be to see whether more nuts per tree will also relate to more nuts in total, and possibly this will have some more complex relationship with the squirrel weight and cheek, and other factors, like high variation in nuts per tree may help to get also an indication about the total number of trees used by a squirrel)

*In phase 2 you will make an estimate of the parameters that are needed to make predictions with the model that has been created in phase 1. The parameters that describe the distribution for the number of nuts per tree can be estimated from the sample measured at the subset of trees. 
A simple way would be to ignore the tree id's and just use the data per squirrel to estimate the distribution parameters and put them into the model from phase 1.
A more precise
model would treat the tree id's as a random factor such that the behavior
specifically attributed to the squirrel can be better estimated. To
treat the trees as a random factor you will have to know how the
trees can be a random factor. You can make an educated guess for
this, but you could also try to learn this from the data (I'd say with some exploratory analysis first, checking out the correlation between a tree and how many nuts get stored in it per squirrel, and whether this effect is independent or maybe some trees attract specific type of squirrels, before coming up with something quantitative.). In the
phase 1 you do not observe information related to the tree ids but in
phase 2 you do and you can use that data.
So in a nutshell. I think you need some exploratory analysis before you can actually do something quantitative that is more than the simple approach (simple is ignoring the tree ids in phase 2 and using just simple distribution parameters as input for the model whose coefficients are learned in phase 1).

Furthermore, how can we introduce partially measured outputs into the training set?
When you make the model in 1 by using parameters that describe the distribution of the nuts per tree for a specific squirrel you need to take into account that it must be possible to reasonably estimate those numbers in phase 2 and that errors will not effect the model too much. For instance mean and variance (or other simple statistics) could be reasonably estimated from the samples in phase 2 (assuming your sample is not too small) but higher order moments may not. 
