4
$\begingroup$

In data science, Which process should come first, handling missing data or handling data types. I am asking this question because I have problem in following cases:

1) Handling Missing data first, then handling data types - It would be difficult to handle missing data for categorical columns

2) Handling data types first, then handling missing data - if we create dummies for catergorical values, it might happen that while handling missing data, we might assign two different categories to same row.

$\endgroup$
6
$\begingroup$

Handle data first, then perform multiple imputation.

Several solid multiple imputation using chained equations (MICE) implementations that I can think of permit contingent imputation where:

  • Specific data types produce specific models, so the quality of your imputation depends on handling data types
  • Interdependence between variables (e.g., mutually exclusive categories) can be explicitly modeled (e.g., using ordered logit or unordered multiple logit)
  • Hard dependencies (e.g., do not impute $x$ and $x^{2}$, but only, for example, impute $x$ using chained equations, and simply calculate $x^{2}$ based on imputed values of $x$, or vice versa)

In this way you can handle missing data for categorical variables alongside continuous or interval variables.

References

Azur, M. J., Stuart, E. A., Frangakis, C., & Leaf, P. J. (2011). Multiple imputation by chained equations: What is it and how does it work? International Journal of Methods in Psychiatric Research, 20(1), 40–49.

White, I. R., Royston, P., & Wood, A. M. (2011). Multiple imputation using chained equations: Issues and guidance for practice. Statistics in Medicine, 30(4), 377–399.

$\endgroup$
1
$\begingroup$

I don't think it matters which one you do first as long as your methods are valid. Yes, imputing categorical missing data is difficult. However, if you create your dummy variables and then try to impute you haven't made your problem any easier as you now have C (number of categories) variables to determine.

I think I would personally take the first approach and try to impute what I can (if that is the route you are taking). MICE is a very flexible method for imputing data of different types.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.