1
$\begingroup$

I am trying to figure out how best to encode ICD10 codes for input into a machine learning model.

It isn't ordinal by any means, however, there is a sort of logic you can apply to just the labels that will tell you which ones can be grouped together without any additional knowledge. In some ways, it's closer to an interval data type in that the difference between consecutive codes frequently (but by no means always) has some kind of meaning.

This makes me think that binary encoding may be more effective than one-hot, but are there other options I should be considering?

for example:

I71.1 Thoracic aortic aneurysm, ruptured

I71.2 Thoracic aortic aneurysm, without mention of rupture

I71.3 Abdominal aortic aneurysm, ruptured

I71.4 Abdominal aortic aneurysm, without mention of rupture

J12.2 Parainfluenza virus pneumonia

J12.3 Human metapneumovirus pneumonia

J12.8 Other viral pneumonia

J12.9 Viral pneumonia, unspecified

$\endgroup$
  • $\begingroup$ Can you explain the difference between binary and one-hot for your application? It seems that one-hot coding, which used to be called dummy indicators, is probably your best option. $\endgroup$ – Todd D Nov 22 '18 at 0:09
  • $\begingroup$ for binary I mean that I would assign a number to each label e.g. J12.3=0, J12.8=1, J12.9=3 and then binary encode these 000, 001, 010 etc. with the idea that this may outperform purely 1-hot because then all the J1 labels would have the higher order digits the same and all the I71 labels would have higher order digits the same as well, allowing groupings to be learned as well as just the actual label - if I predict 'other viral pneumonia' when it should have been one of the other choices, this is still better than many other predictions despite not being 100% correct $\endgroup$ – gkennos Nov 22 '18 at 0:14
  • $\begingroup$ Also I am intending to drop digits to the right of the decimal point to improve on sparsity, but included all here for example $\endgroup$ – gkennos Nov 22 '18 at 0:15
  • $\begingroup$ Are you training a model to cluster the codes or using the codes as explanatory features/variables? $\endgroup$ – Todd D Nov 22 '18 at 0:22
  • $\begingroup$ as explanatory features $\endgroup$ – gkennos Nov 22 '18 at 0:27
1
$\begingroup$

You should encode as dummy variables ("one-hot"). I don't understand how you could use your binary encoding, you would have to explain. But you say In some ways, it's closer to an interval data type in that the difference between consecutive codes frequently (but by no means always) has some kind of meaning. This is interesting, but probably difficult to encode information. Maybe there is an Bayesian way. But this could still be exploited, and you say one goal (additional) could be to learn that some labels are close to other labels in effect.

One method which could achieve that, is fused lasso, search this site or look at Principled way of collapsing categorical variables with many levels? or Continuous dependent variable with ordinal independent variable

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.