Let me explain what I mean by unfairness.

Let's say I have a multi-class classification problem where I am trying to predict the 'best drug' (among multiple candidate drugs) for each patient. So for each entry (patient) in my database, I have recorded my target variable which is the 'best drug'.

The problem is that each patient didn't try all the candidate drugs but only a smaller set of them (set that I didn't record). The only thing I know is the 'best' one among this smaller set, and what I would like to predict is the best one among all the possible drugs.

I wanted if you knew any techniques or models that can handle this type of 'unfairness'?

  • 2
    $\begingroup$ The technical term is missing data. $\endgroup$ – Neil G Oct 5 '18 at 20:58
  • $\begingroup$ per se all the features and target are filled so it is not the typical missing data? $\endgroup$ – user130104 Oct 5 '18 at 22:16
  • 2
    $\begingroup$ You're missing the observations of some patients-drug combinations. $\endgroup$ – Neil G Oct 5 '18 at 22:29
  • $\begingroup$ (Un)Fairness in ML refers to something else usually... $\endgroup$ – usεr11852 Oct 5 '18 at 22:50
  • $\begingroup$ @NeilG you are right, I am actually missing all the patient-drug combinations except the one who worked 'best' in the little sample we tried $\endgroup$ – user130104 Oct 6 '18 at 15:20

If the list of drugs tested by a patient is a random variable that does not depend on the patient, I suggest that you start by clustering your patients. I assume that patients from a single cluster are likely to behave the same with respect to each drug. Then, from each cluster, the "best" drug would be the one occurring the most. The main issue here is to make clusters big enough to have sufficient data in each, but not too big in order not to mix different populations.

If the list of tested drugs actually depends on the patient characteristics, this seems more complicated. Here's a basic idea from which you may be able to develop a more sophisticated approach: you could treat your problem by adding a list of "hidden" features (one for each drug) with missing data, telling if the patient has tested the drug or not. The one they picked as the best one has been tested for sure, the corresponding feature would then be 1. About the other drugs, you can assign the "this drug has been tested" feature a score that depends on the fraction of similar patients (from the same cluster) who selected this drug as the best one.

Edit: I put this post on community wiki mode so that someone can help from that point.

  • $\begingroup$ Thanks! There are usually a small pool of drugs always tried first on the patients, but if they don't work well enough stronger drugs are tried - often depending on the characteristics of the patients. So if I follow your technique I would put 1 to this small pool and a score relative to the cluster the patient belongs to for the remaining drugs? $\endgroup$ – user130104 Oct 6 '18 at 15:43
  • $\begingroup$ This would be right. Thus you can complete your missing data. Then, for each patient, you will be able to tell which drug is the best among a likely subset. This will mean that the best is either the one selected by the patient, or one that is unlikely to have been tested on them. The problem is not finished but you can surely go on with this, as you have more detailed information. $\endgroup$ – Romain Reboulleau Oct 6 '18 at 16:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.