2
$\begingroup$

There are a lot of threads on here about missing data, but I haven't found something that really gets at the best practices, and discussion of why to choose one approach over another. This is such a common topic in any introduction to machine learning that I've seen, but there doesn't appear to be clear consensus on what to do. Is there a robust approach works pretty well in practice? Can we automate how we deal with missing data? Is there a reason there isn't?

I'll list a few common answers to this question I see, and a couple more robust, less common approaches.

There are some common recommendations including:

  1. getting rid of rows with missing data;
  2. getting rid of columns (features) with missing data;
  3. filling in missing values with mean, median, or mode;
  4. use some nearest neighbors imputation;

These seem like ways to quickly get a model out the door, but don't seem realistic in practice. You will have a lot of messy data in the real world. Lots of missing data for lots of missing features. And a lot of times missing data can be a great predictor for things. Moreover, there's a strong emotional repulsion I have to disregarding data, or changing it. Every fiber of my being says this is a terrible idea. Here are some more common procedures: https://www.ncbi.nlm.nih.gov/pubmed/23853744

I went through a data science bootcamp and our instructors recommended a more thorough approach if data is really messy which I like quite a bit more:

  1. Create a new feature that is an indicator function for each feature being populated or not.
  2. Then impute the missing values to some statistic like the mean, so that our ML algorithm will not have issues with missing data

This makes a lot of sense to me, but there's the disadvantage of doubling your feature space. How bad is this in practice? Are there other disadvantages?

I'm also taking an ML classification class mooc and it recommends a different approach in the context of decision trees. At each split, have the algorithm lump missing values with another group. So if we're on a feature with categories A,B,C...we can split this feature a few different ways:

  1. A or missing,B, C
  2. A, B or missing, C
  3. A, B ,C or missing

This approach seems similar to the approach above, but has some practical drawbacks in rewriting every algorithm we use. Does this approach provide any benefits over the above? Are there other robust methods that compete with these approaches?

$\endgroup$

closed as too broad by Peter Flom Oct 14 '18 at 12:47

Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

  • 1
    $\begingroup$ I'm afraid there's no answer of a couple of screens' worth that will tie together all the best practices and answer all of your questions on this. Perhaps you are motivated enough to tackle one of the books on the subject; Paul D. Allison wrote a small introductory one, for example. Otherwise I recommend looking into multiple imputation as more comprehensive than nearest neighbors and as an effective strategy if data are at least missing at random ("MAR" or "MCAR"). $\endgroup$ – rolando2 Oct 14 '18 at 12:28
  • $\begingroup$ Is econd @rolando2 recommendation of considering Multiple Imputation, the trouble is its really hard to demonstrate data is Missing At Random (MAR) or Missing Completely At Random (MCAR). You may find this dsicussion useful. I personally would rather use complete data, but recognise this isn't always practical and would perform an analysis on such a subset data to see how it compares to other methods. $\endgroup$ – slackline Oct 14 '18 at 12:32
  • $\begingroup$ A good short overview is John W. Graham (2009), "Missing data analysis: Making it work in the real world". $\endgroup$ – rolando2 Oct 14 '18 at 12:44
  • 1
    $\begingroup$ In addition to your question being very broad (as the other answers indicate), of the four "recommended" methods you list, the first three are, in everything I've read, almost never recommended except in special cases. $\endgroup$ – Peter Flom Oct 14 '18 at 12:47