I am solving a classification problem. The final dataset has around 50 - 60 features and around 12 K - 15 K rows. Each distinct row representing distinct ID. Now the problem is that for each of these features, most of the rows are empty/missing values. That's because these 50 - 60 features represent all the signals in the environment. And hence all the IDs doesn't use all the features but rather only a set of features. And to take care of missing values, currently I am filling them with "zeros". So this is how the dataset looks like[As an example]:

ID   F1   F2    F3    F4       Label
1    10   0     40    0         A
2    30   20    0     200000    B
3    0    0     30    100000    A
4    0    90    100   0         B
5    0    0     0     0         A
6    0    0     0     300000    B
7    0    600   0     0         A
8    0    300   0     150000    B
9    40   0     300   0         A
10   100  56    79    0         B

Now I am using a Decision Tree model to predict the labels. And I getting a F-score of around 50-55 in prediction. How can I tackle the problem mentioned above and improve the F-score. I will be very relaxed if I get some good solution for this. Thanks a ton in advance.

  • $\begingroup$ Yes,I am still working on them.I am trying to implement them and then accordingly I will update my responses here. Moreover @SandeepS.Sandhu , your answer gives a good overview of the answer but I was looking for something more detailed. Point 3 of yours, is most imp for me and that is explained with more details in tomka's answer.. $\endgroup$ Sep 19 '16 at 11:16

I suggest you could do the following:

  1. Do not replace missing values with 0 (zeros).
  2. In case all features are missing for a particular record (e.g. row id 5), then remove that row from the training set.
  3. If only a few features (or just 1 feature) is missing, then you could try imputing the missing values. This will help you use many more rows which you would have filtered out otherwise.

Filling in zeros almost certainly will bias your estimates, except in the special case where all missing cases are plausible zeros.

Deleting cases listwise assumes that missing data occur completely at random (MCAR), thus do not depend on any of the observed information. This is often also implausible and if data are MCAR your estimates will be likewise biased.

You could therefore try imputing missing values, assuming the data are missing at random (MAR) conditionally on the observed information. There are many ways how you could do this in practice. The best option is to use a Bayesian multiple imputation procedure. It will impute multiple times creating multiple imputed data sets. The uncertainty in imputed values is reflected by variance in imputations across data sets. For literature on this topic see Schafer and Graham (2002) or Little and Rubin (2001) or the reference work of Rubin (1987).

For an hands on approach you could use multiple imputation by chained equations, see van Buuren (2012). There is an R package called mice where this procedure has been implemented. It is the most straight forward approach for handling the missing data problem you are facing.

One final note, in case you find working with multiply imputed data sets too complicated I suggest you start with $m=1$ in mice, which will only impute once.

Little, R. J. A., & Rubin, D. B. (2002). Statistical analysis with missing data (2nd ed.). Hoboken: Wiley.

Rubin, D. B. (1987). Multiple Imputation for Nonresponse in Surveys. New York: John Wiley & Sons.

Schafer, J. L., & Graham, J. W. (2002). Missing data: Our view of the state of the art. Psychological Methods, 7(2), 147–177. http://doi.org/10.1037/1082-989X.7.2.147

van Buuren, S. (2012). Flexible Imputation of Missing Data. Boca Raton: CRC Press.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.