1
$\begingroup$

Suppose you have the classical example of predicting house prices and you have as input features area size, built year, number of previous owners, city, number of floors, number of bedrooms, etc. But in this case, the expected outputs are the sale price, the rent price and annual tax.

Now suppose you have a lot of input data but one or more features are missing, also not all the outputs (labels) are present. For example:

| Area |  Year | owners | rooms || sale | rent | tax |
------------------------------------------------------
|   80 |  2001 |      X |     4 || 250k |    X | 200 |
|   70 |  2015 |      X |     2 || 125k | 1200 |   X |
   ...
|   60 |  1995 |      4 |     2 ||    X | 1000 | 100 |
|    X |     X |      2 |     3 || 300k | 1600 | 330 |
   ...
|    X |     X |      1 |     X || 220k |    X |   X |
|    X |  2005 |      2 |     X ||    X |    X | 150 |

A possible scenario I think for this could be gathering data from different sources like real state companies, Government entities, self-published post by owners, etc. (Different datasets with overlapping features and outputs)

I think that having some amount of this incomplete data is enough to generate a model to estimate the outputs.

I have found some approaches like filling the missing values with mean or zero. Or discarding rows with missing values. But I'm looking more something like ignoring some part of the model when an input is not present or guess the value based on the dependency with other features, but this is not needed.

What kind of network could be used or what training approach should be used to address this type of problem? Any Tensorflow resource related?

NOTE: This question is closely related, but after 8 years I hope we have new approaches to try.

NOTE 2: I originally posted this question on StackOverflow and was deleted, but I post below the comments I got:

  • You may try with a non-fully connected neural network, if some rows of your dataset are not fully filled, or you can try with dimensionality reduction to determine which columns to keep, if some of them are not fully filled. – Daniel R. Oct 4 at 16:54

  • One option you didn't mention is to add a separate input for 'unknown' and let the NN learn what to do with it. But it seems more like a question for datascience.stackexchange.com – de1 Oct 4 at 17:54

  • Thanks for your comments. @DanielR., do you have any resource on this non-fully connected NN? I don't consider dimensionality reduction yet because most of the rows have missing fields, if not all. – GVelascoh Oct 5 at 8:18

  • @de1 do you mean something like having an additional binary input indicating if certain feature exist? If I have N features, use 2N features (N originals + N binaries)? Any resource on this approach? – GVelascoh Oct 5 at 8:18

  • It was an intution, this paper might be related but didn't have time to study it: link There is also a discussion here: link As mentioned I think it might be better to move the question as stackoverflow is meant more for precise programming questions. Your question is more open ended, there may be current good approaches. But they depend onn your data and your model. New approaches may be developed in the future. A moving target. – de1 Oct 5 at 8:54

  • I read this paper on sparse neural networks, link and at page 317 (3 of the pdf) they state that if a representation is both sparse and robust to small input changes, the set of non-zero features is almost always roughly conserved, and also that Different inputs may contain different amounts of information and would be more conveniently represented using a variable-size data-structure. You will have to build your model from scratch though, as I am not aware of any premade ones. Your NaN values would then count as 0's. – Daniel R. Oct 5 at 16:10

$\endgroup$
  • $\begingroup$ An approach often used is to take the mean of the values. If you have a categorical variable you could also specify a separate category which would have the same effect. Sometimes it is better to fill in a 0, but this can also go very wrong if we have a linear function. So the more complicated answer is: it depends. $\endgroup$ – Jan Oct 18 '18 at 14:52
0
$\begingroup$

The general approach is to fill (pad) those values with some other neutral value. But you said you have tried that.

You're asking if you could shut down a part of the model that is responsible for some feature if that feature is not present in the example. I don't think it's possible.

guess the value based on the dependency with other features

This would mean to have a different model to predict the value of the feature based on other features right? But I think that approach is not very good if the number of features that are missing is high because you'd have to have a lot of models for a lot of combinations of features. Maybe something that you can do is have a different model based on the source where you have found the data.

I also guess, since you're asking this question, that the shared set of features is not big enough for your predictions to be relevant?

$\endgroup$
  • $\begingroup$ Thanks for your answer. I updated the question with some details $\endgroup$ – gustavovelascoh Oct 18 '18 at 15:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.