I need help with planning and ideas on how to build an data table for a machine learning model in sklearn.
The model is supposed to help the clinic manager decide which treatment method and therapist best fits a specific patient according to past experience with other patients.
The input that the model will receive in prediction is the patient's details (personal details, etc.) the model should propose the most appropriate treatment method and therapist.
The data I have to train the model is:

  1. patient details
  2. selected treatment method
  3. selected therapist
  4. feedback on the patient's progress (how the patient progressed / retreated)

My question is how to organize the table - which columns will be the attributes? What will be the tag? And how do I get classification for both preferred treatment method and therapist? (Allegedly there are 2 tags here)
We would be happy to receive any ideas


thanks for the reply

  1. In the first stage, all use of the model will be at the same time together with human judgment and decision. And not in their place. So that significant implications will not be at this stage.

  2. In order not to rely on incorrect previous data, I have a column in the table that shows what the success of the specific treatment was - and of course I would like the model to give a significant ratio to this column.

I have some ideas on how to do this but all of them have "bugs" at certain points.
My main question is how to decide what will be the target column in the table and then the rest of course will be the attributes.

whether the method and the therapist or the progress - and then in model will be a regression model. (The progress column is a numerical value)

something else -I have not found in my search examples of using a double target- I would be happy if you can direct me to code samples (if any)


Generally the attributes, or features, could be the patient details and the tag, or target, could be the treatment method and therapist. So the target for each patient would be a 2x1 vector, where one element corresponds to a treatment method and the other to a therapist.

This would be easy to implement in sklearn, it is common to have a 2-dimensional target, but I am not sure it would be appropriate for your goals.

I am assuming this is observational data. Suppose you train this model to predict the therapist and treatment. The question is whether it helps. For example, what if the commonly prescribed treatment is actually worse for the patient? Since that treatment is common, the model would learn to prescribe that treatment anyway.

If however you believe that the data represents good decisions, then the model you build should be alright. An example of this might be differentiating patients who need to see a pediatrician from patients who need to see a geriatrician. If there is a very clear distinction between these two patients (very high signal to noise ratio), and in the past generally these patients have been classified correctly (as reflected in the data), then the model should be alright. In this case, the patient detail would be age and it should fairly easily help one decide which provider to send to, and probably in the past (in your data) not too many people have gotten this wrong. So, it should be alright.

Conversely, if the question is whether to send to a cardiologist or a neurologist, the signal to noise ratio is much lower. Suppose the patient detail is symptom -- dizziness. This could be either brain or heart related, and often it will be so to varying degrees. In this case, the data might be full of cases where a patient was sent to a neurologist, but should have gone to a cardiologist, and vice versa. Perhaps, even, the data is overwhelmed by cases of patients who were wrongly sent to a neurologist. In this case, the model would learn to send patients who had to see a cardiologist to a neurologist, and would make care worse.

This becomes even more complicated when you have two targets, especially when the data is observational, and, as in your case, it involves interventions. Clearly you have more information than just the treatment and therapist since you also have information on the patient's progression. You could use that information to create a better model. But I think that the complexity of implementing such a model in sklearn will pale in comparison to the complexity of defining your goals in the context of the data.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.