Suppose you have the classical example of predicting house prices and you have as input features area size, built year, number of previous owners, city, number of floors, number of bedrooms, etc. But in this case, the expected outputs are the sale price, the rent price and annual tax.
Now suppose you have a lot of input data but one or more features are missing, also not all the outputs (labels) are present. For example:
| Area | Year | owners | rooms || sale | rent | tax | ------------------------------------------------------ | 80 | 2001 | X | 4 || 250k | X | 200 | | 70 | 2015 | X | 2 || 125k | 1200 | X | ... | 60 | 1995 | 4 | 2 || X | 1000 | 100 | | X | X | 2 | 3 || 300k | 1600 | 330 | ... | X | X | 1 | X || 220k | X | X | | X | 2005 | 2 | X || X | X | 150 |
A possible scenario I think for this could be gathering data from different sources like real state companies, Government entities, self-published post by owners, etc. (Different datasets with overlapping features and outputs)
I think that having some amount of this incomplete data is enough to generate a model to estimate the outputs.
I have found some approaches like filling the missing values with mean or with zero. Or discarding rows with missing values. But I'm looking more something like ignoring some part of the model when an input is not present or guess the value based on the dependency with other features, but this is not needed.
What kind of model could be used or what training approach should be used to address this type of problem?
NOTE: This question is closely related, but after 8 years I hope we have new approaches to try.
NOTE 2: I originally posted this question on StackOverflow and was deleted, but I post below the comments I got:
You may try with a non-fully connected neural network, if some rows of your dataset are not fully filled, or you can try with dimensionality reduction to determine which columns to keep, if some of them are not fully filled. – Daniel R. Oct 4 at 16:54
One option you didn't mention is to add a separate input for 'unknown' and let the NN learn what to do with it. But it seems more like a question for datascience.stackexchange.com – de1 Oct 4 at 17:54
Thanks for your comments. @DanielR., do you have any resource on this non-fully connected NN? I don't consider dimensionality reduction yet because most of the rows have missing fields, if not all. – GVelascoh Oct 5 at 8:18
@de1 do you mean something like having an additional binary input indicating if certain feature exist? If I have N features, use 2N features (N originals + N binaries)? Any resource on this approach? – GVelascoh Oct 5 at 8:18
It was an intution, this paper might be related but didn't have time to study it: link There is also a discussion here: link As mentioned I think it might be better to move the question as stackoverflow is meant more for precise programming questions. Your question is more open ended, there may be current good approaches. But they depend onn your data and your model. New approaches may be developed in the future. A moving target. – de1 Oct 5 at 8:54
I read this paper on sparse neural networks, link and at page 317 (3 of the pdf) they state that if a representation is both sparse and robust to small input changes, the set of non-zero features is almost always roughly conserved, and also that Different inputs may contain different amounts of information and would be more conveniently represented using a variable-size data-structure. You will have to build your model from scratch though, as I am not aware of any premade ones. Your NaN values would then count as 0's. – Daniel R. Oct 5 at 16:10