ML technique recommendations where each feature has multiple properties and the number of features per observation varies Here is an abstracted version of the problem I am facing.
I wish to predict the true value of a variable from multiple noisy predictions as to its value. There are three complicating factors:
For each observation, I receive multiple noisy predictions from different sources. The sources may each provide valuable information, but are likely to differ in their accuracy (I'm using accuracy in its common sense rather than as a technical term here).
Each prediction is timestamped, with later predictions tending to be more accurate than earlier ones. The function which defines the relationship between time and accuracy is strictly increasing but otherwise unknown (e.g. it may be discontinuous, the order rather than the amount of time elapsed may be the only thing that matters).
The number of predictions obtained from any given source is not constant across observations (and the number of predictions for any given observation varies from source to source).
Just to provide a very rough illustration:
obs 1:
y = 2.5,
x[t=10,s=1] = 2.6,
x[t=7,s=3] = 2.2,
x[t=4,s=1] = 2.3,
x[t=3,s=3] = 3.1
obs 2:
y = 4.2,
x[t=8,s=3] = 4.5,
x[t=3,s=1] = 3.7,
x[t=2,s=1] = 3.8
x[t=1,s=2] = 3.8
etc
So what do I do guys? I'm having real trouble seeing how to translate this data into a table of features. The main problem I can see is that the source, timestamp, and value for each observation of x seem to be inseparable and thus ought to be coded as a single feature. But this appears to be impossible.
I suppose modelling each source separately and ensembling the models would be easier, but it doesn't seem ideal. For instance, even if source 1 is generally more accurate, it shouldn't be weighted as highly if its predictions are on average earlier for a given observation.
I imagine that some kind of neural network would be best at aggregating these data in such a way as to take advantage of all the potential non-linearities, but I'm struggling to work out would the best algorithm would be and how to engineer features for it.
 A: Try this for the x samples:


*

*Treat the indexes of x (time and source) and its values as separate columns (e.g. x now has examples of 3 fields like (t,s_t,x_t)).

*Discretize your timestamps (e.g. granularity of 1 sec ou 1 min).

*At the instants t that don't have data, fill the fields with the mean of the other examples. A worse but easier workaround is filling with dummy values (e.g. zero).


For y:


*

*Use LSTM to map from x but take only the last output to compute the result (e.g. add a regression layer at the end)
OR

*Use LSTM and aggregate all outputs (e.g. with mean and variance) and do regression with the result (e.g. with regression layer at the end)

A: You do not provide much detail in your question, so the answer is likewise somewhat vague. What is clear is that you have a panel data structure. Check Wikipedia for how this is usually formatted. Given that your observations are unequally spaced, you will have many missing data points. There are two solutions to this:


*

*Reduce missing values by downsampling the observed data, which obviously results in information loss.

*Create new data that resembles the observed data with a (multiple) imputation process. This does not result in information loss, but may seriously bias the results. How serious depends on many factors.


But in the example that you describe, there may be another interesting solution. If the sensors all measure the same thing, but with differing accuracy, then they can represented as measurements of a single sensor with a varying (but known) accuracy. This collapses one dimension of the panel, thus reducing the amount of missing data. But no information needs to be lost because the accuracy of the different sensors can be encoded as sampling weights.
A: I'm a bit of a one-trick pony lately, but this seems like a perfect case for Bayesian methods. You need incremental information, and the fact that the observations are noisy further supports the case for them.
Your goal would be to get $p(Y|S,P,X, [T])$, where the parameters are: 
$S$ - Source
$P$ - Precision (i.e. reliability of the source)
$X$ - Value measured by the Source.
$T$ - Time (optional, and will complicate matters)
For multiple sources, you can then combine these into the final posterior distribution via the same general procedure.
In particular, if you ignore Time and your sources are some physical measurements (or are otherwise very macro-scale), it should be safe to model them all as Gaussians. This would let you plug in the numbers to an already established equations in a fairly straightforward fashion. 
To spare some virtual trees, and since I don't know if you know the source precision or not, you can find them here.
