0
$\begingroup$

I am attempting to cluster subgroups of substance abuser based on diagnostic status (nominal), age of onset (ordinal since it is binned in our set), etc. My question regards how to treat missing data for non-users. Obviously, these are subjects who will have no value for "age of onset" since they have no age where they started drinking (for example).

I was planning to use the Daisy package in R to calculate the Gower distance which would allow use of nominal and ordinal variables in distance computation. However, the ordinal variables like age of onset, number of days substances were used in the last week, and dosage of substance used by definition have missing values for non-users. We still need to treat non-users in a sensible way though, because one major prediction of this analysis is that clustering on the Gower distance will separate out one group of non-users and multiple subtypes of substance user.

If the question is not clear enough please let me know any additional information that might help.

$\endgroup$
  • $\begingroup$ My first thought is to use a Bayesian generative model. Assume there is some latent (continuous) variable z that emits the joint observations $c$ (categorical), $x$:(continuous) in a stepwise fashion such that $P(c,x|z)=P(x|c,z)P(c|z)$. Note this is just for convenience, not a statement of causality. Treat the missing data as if it comes from a uniform conditional distribution $P(x_i|c_i,z)$. In practice, you can just omit that contribution to the (log-)posterior as it will always be constant. You could then cluster the $z$ by attaching some kind of Bayesian cluster generator to the model. $\endgroup$ – deasmhumnha Oct 29 '19 at 2:29
0
$\begingroup$

Just split your data into users and non-users, because you don't want them in the same cluster anyway.

Your bigger problem is that your attributes have very different scale. And while Gower's has an built-in heuristic for scaling, you will never know how reliable your results are, if they end up being usable at all.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ This could be a possible answer for a datset containing only one substance that people use, but we have data on ten-ish classes of substance. For example, very few people are users of opioids, but most are users of alcohol. It is very likely that based on (perhaps) a light use profile, that many alcohol users should get clustered with those who use no substances at all. Based on the issues you bring up though, it's likely we'll need to find a very different method for forming functional subgroups of individuals in this sample. $\endgroup$ – Eric Rawls Oct 28 '19 at 14:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.