0
$\begingroup$

I am part of a team that is creating an app to accompany stroke patients through the recovery process. One component of this is creating an algorithm to suggest treatments based on certain clinical data. I have some clinical data from real and in silico patients where we record the following inputs:

  • Motor impairment (Fugl-Meyer upper extremity or FMUE) scores on 33 exercises, which can be 0 (full impairment), 1 (partial impairment), or 2 (no impairment): FMUE_1,...,FMUE_33
  • Spasticity (Modified Ashworth scale or MAS) scores for finger flexion, finger extension, wrist flexion, wrist extension, elbow flexion, elbow extension, shoulder anteversion, shoulder retroversion, shoulder abduction, shoulder adduction, shoulder inner rotation, shoulder outer rotation. Each score is chosen from the list 0 (muscle is fully flexible), 1, 1.5, 2, 3, 4 (muscle is fully rigid)
  • Depression (Hospital Anxiety and Depression Scale-Depression) scores on 7 questions, each of which is answered with 0 (totally agree), 1, 2, 3 (totally disagree): HADS_D_1,...,HADS_D_7
  • Working memory (Corsi score): a score ranging from 0 (no memory) to 9 (very good memory)
  • Shoulder pain (Visual Assessment Scale or VAS): a score ranging from 0 (no pain),...,10 (worst pain imaginable)
  • Lifetime prevalence of epilepsy: 0 (never had a seizure), 1 (ever had a seizure)
  • Willingness to consider experimental treatments: 0 (not willing), 1 (willing)

I envision some feature engineering, as follows.

  1. There would be a "proximality index" telling whether the motor impairment is chiefly proximal (shoulder, elbow), chiefly distal (wrist, fingers) or proximal and distal in roughly equal measure. FMUE scores 1 thru 18 measure proximal impairment while FMUE scores 19 thru 33 measure distal impairment, so the proximality index would be the average of (FMUE_18,...,FMUE_33) minus the average of (FMUE_1,...,FMUE_18). Using the proximality index, we would define two boolean values: PROXIMAL, which is 1 if proximality index is greater than 0.2 and 0 otherwise; and DISTAL, which is 1 if proximality index is less than -0.2 and 0 otherwise. Note that these two booleans can both be 0 (if the proximality index is between -0.2 and 0.2) but they cannot both be 1.
  2. We would assess overall motor impairment by defining FMUE to be the sum of (FMUE_1,...,FMUE_33).
  3. We would assess overall spasticity by defining MAS to be the maximum of (finger flexion,...,shoulder outer rotation).
  4. We would assess overall depression by defining HADS_D to be the sum of (HADS_D_1,...,HADS_D_7).
  5. The features CORSI, VAS, EXPERIMENTAL and EPILEPSY remain unchanged.

The training label for each patient is a subset of the following set of treatments:

  • TIMES (an experimental treatment)
  • ECOSS (another experimental treatment)
  • AVANCER (yet another experimental treatment)
  • CIMT (constraint induced movement therapy)
  • PROXIMAL_ES (electrical stimulation to proximal muscles)
  • DISTAL_ES (electrical stimulation to distal muscles)
  • MIRROR (mirror therapy, where one uses a mirror to trick the brain into thinking that the impaired limb works as well as the healthy one)
  • PSYCHOTHERAPY (to treat depression)

To ensure maximum explainability, I believe that a multilabel decision tree is the way to go. Using scikit-learn, I have created a pipeline of feature engineering followed by the classifier and successfully trained it. The problem is that I also need to export the model to PMML to give to the production team. I tried sklearn2pmml and when that failed, I decided to try to write out the PMML file from scratch. Unfortunately, that too is proving harder than I thought. Here's what I have so far:

<PMML version="4.4">
  <Header/>
  <DataDictionary>
    <!-- non-preprocessed fields -->
    <DataField name="FMUE_1" optype="continuous" dataType="double"/>
    <DataField name="FMUE_2" optype="continuous" dataType="double"/>
    <DataField name="FMUE_3" optype="continuous" dataType="double"/>
    <DataField name="FMUE_4" optype="continuous" dataType="double"/>
    <DataField name="FMUE_5" optype="continuous" dataType="double"/>
    <DataField name="FMUE_6" optype="continuous" dataType="double"/>
    <DataField name="FMUE_7" optype="continuous" dataType="double"/>
    <DataField name="FMUE_8" optype="continuous" dataType="double"/>
    <DataField name="FMUE_9" optype="continuous" dataType="double"/>
    <DataField name="FMUE_10" optype="continuous" dataType="double"/>
    <DataField name="FMUE_11" optype="continuous" dataType="double"/>
    <DataField name="FMUE_12" optype="continuous" dataType="double"/>
    <DataField name="FMUE_13" optype="continuous" dataType="double"/>
    <DataField name="FMUE_14" optype="continuous" dataType="double"/>
    <DataField name="FMUE_15" optype="continuous" dataType="double"/>
    <DataField name="FMUE_16" optype="continuous" dataType="double"/>
    <DataField name="FMUE_17" optype="continuous" dataType="double"/>
    <DataField name="FMUE_18" optype="continuous" dataType="double"/>
    <DataField name="FMUE_19" optype="continuous" dataType="double"/>
    <DataField name="FMUE_20" optype="continuous" dataType="double"/>
    <DataField name="FMUE_21" optype="continuous" dataType="double"/>
    <DataField name="FMUE_22" optype="continuous" dataType="double"/>
    <DataField name="FMUE_23" optype="continuous" dataType="double"/>
    <DataField name="FMUE_24" optype="continuous" dataType="double"/>
    <DataField name="FMUE_25" optype="continuous" dataType="double"/>
    <DataField name="FMUE_26" optype="continuous" dataType="double"/>
    <DataField name="FMUE_27" optype="continuous" dataType="double"/>
    <DataField name="FMUE_28" optype="continuous" dataType="double"/>
    <DataField name="FMUE_29" optype="continuous" dataType="double"/>
    <DataField name="FMUE_30" optype="continuous" dataType="double"/>
    <DataField name="FMUE_31" optype="continuous" dataType="double"/>
    <DataField name="FMUE_32" optype="continuous" dataType="double"/>
    <DataField name="FMUE_33" optype="continuous" dataType="double"/>
    <DataField name="FINGER_FLEXION_MAS" optype="continuous" dataType="double"/>
    <DataField name="FINGER_EXTENSION_MAS" optype="continuous" dataType="double"/>
    <DataField name="WRIST_FLEXION_MAS" optype="continuous" dataType="double"/>
    <DataField name="WRIST_EXTENSION_MAS" optype="continuous" dataType="double"/>
    <DataField name="ELBOW_FLEXION_MAS" optype="continuous" dataType="double"/>
    <DataField name="ELBOW_EXTENSION_MAS" optype="continuous" dataType="double"/>
    <DataField name="SHOULDER_ANTEVERSION_MAS" optype="continuous" dataType="double"/>
    <DataField name="SHOULDER_RETROVERSION_MAS" optype="continuous" dataType="double"/>
    <DataField name="SHOULDER_ABDUCTION_MAS" optype="continuous" dataType="double"/>
    <DataField name="SHOULDER_ADDUCTION_MAS" optype="continuous" dataType="double"/>
    <DataField name="SHOULDER_INNER_ROTATION" optype="continuous" dataType="double"/>
    <DataField name="SHOULDER_OUTER_ROTATION" optype="continuous" dataType="double"/>
    <DataField name="HADS_D_1" optype="continuous" dataType="double"/>
    <DataField name="HADS_D_2" optype="continuous" dataType="double"/>
    <DataField name="HADS_D_3" optype="continuous" dataType="double"/>
    <DataField name="HADS_D_4" optype="continuous" dataType="double"/>
    <DataField name="HADS_D_5" optype="continuous" dataType="double"/>
    <DataField name="HADS_D_6" optype="continuous" dataType="double"/>
    <DataField name="HADS_D_7" optype="continuous" dataType="double"/>
    <DataField name="CORSI" optype="continuous" dataType="double"/>
    <DataField name="VAS" optype="continuous" dataType="double"/>
    <DataField name="EPILEPSY" optype="continuous" dataType="double"/>
    <DataField name="EXPERIMENTAL" optype="continuous" dataType="double"/>
    <!-- preprocessed fields -->
    <DataField name="PROXIMAL" optype="continuous" dataType="double"/>
    <DataField name="DISTAL" optype="continuous" dataType="double"/>
    <DataField name="FMUE" optype="continuous" dataType="double"/>
    <DataField name="FMUE" optype="continuous" dataType="double"/>
    <DataField name="MAS" optype="continuous" dataType="double"/>
    <DataField name="HADS_D" optype="continuous" dataType="double"/>
    <!-- labels -->
    <DataField name="class" optype="categorical" dataType="string">
      <Value value="TIMES"/>
      <Value value="ECOSS"/>
      <Value value="AVANCER"/>
      <Value value="CIMT"/>
      <Value value="PROXIMAL_ES"/>
      <Value value="DISTAL_ES"/>
      <Value value="MIRROR"/>
      <Value value="PSYCHOTHERAPY"/>
    </DataField>
  </DataDictionary>
  <TransformationDictionary>
    <DerivedField name="PROXIMAL" optype="continuous" dataType="double">
      <Apply function="threshold">
        <Constant dataType="double">
          -0.2
        </Constant>
        <Apply function="-">
          <Apply function="avg">
            <FieldRef field="FMUE_1"/>
            <FieldRef field="FMUE_2"/>
            <FieldRef field="FMUE_3"/>
            <FieldRef field="FMUE_4"/>
            <FieldRef field="FMUE_5"/>
            <FieldRef field="FMUE_6"/>
            <FieldRef field="FMUE_7"/>
            <FieldRef field="FMUE_8"/>
            <FieldRef field="FMUE_9"/>
            <FieldRef field="FMUE_10"/>
            <FieldRef field="FMUE_11"/>
            <FieldRef field="FMUE_12"/>
            <FieldRef field="FMUE_13"/>
            <FieldRef field="FMUE_14"/>
            <FieldRef field="FMUE_15"/>
            <FieldRef field="FMUE_16"/>
            <FieldRef field="FMUE_17"/>
            <FieldRef field="FMUE_18"/>
          </Apply>
          <Apply function="avg">
            <FieldRef field="FMUE_19"/>
            <FieldRef field="FMUE_20"/>
            <FieldRef field="FMUE_21"/>
            <FieldRef field="FMUE_22"/>
            <FieldRef field="FMUE_23"/>
            <FieldRef field="FMUE_24"/>
            <FieldRef field="FMUE_25"/>
            <FieldRef field="FMUE_26"/>
            <FieldRef field="FMUE_27"/>
            <FieldRef field="FMUE_28"/>
            <FieldRef field="FMUE_29"/>
            <FieldRef field="FMUE_30"/>
            <FieldRef field="FMUE_31"/>
            <FieldRef field="FMUE_32"/>
            <FieldRef field="FMUE_33"/>
          </Apply>
        </Apply>
      </Apply>
    </DerivedField>
    <DerivedField name="DISTAL" optype="continuous" dataType="double">
      <Apply function="threshold">
        <Apply function="-">
          <Apply function="avg">
            <FieldRef field="FMUE_1"/>
            <FieldRef field="FMUE_2"/>
            <FieldRef field="FMUE_3"/>
            <FieldRef field="FMUE_4"/>
            <FieldRef field="FMUE_5"/>
            <FieldRef field="FMUE_6"/>
            <FieldRef field="FMUE_7"/>
            <FieldRef field="FMUE_8"/>
            <FieldRef field="FMUE_9"/>
            <FieldRef field="FMUE_10"/>
            <FieldRef field="FMUE_11"/>
            <FieldRef field="FMUE_12"/>
            <FieldRef field="FMUE_13"/>
            <FieldRef field="FMUE_14"/>
            <FieldRef field="FMUE_15"/>
            <FieldRef field="FMUE_16"/>
            <FieldRef field="FMUE_17"/>
            <FieldRef field="FMUE_18"/>
          </Apply>
          <Apply function="avg">
            <FieldRef field="FMUE_19"/>
            <FieldRef field="FMUE_20"/>
            <FieldRef field="FMUE_21"/>
            <FieldRef field="FMUE_22"/>
            <FieldRef field="FMUE_23"/>
            <FieldRef field="FMUE_24"/>
            <FieldRef field="FMUE_25"/>
            <FieldRef field="FMUE_26"/>
            <FieldRef field="FMUE_27"/>
            <FieldRef field="FMUE_28"/>
            <FieldRef field="FMUE_29"/>
            <FieldRef field="FMUE_30"/>
            <FieldRef field="FMUE_31"/>
            <FieldRef field="FMUE_32"/>
            <FieldRef field="FMUE_33"/>
          </Apply>
        </Apply>
        <Constant dataType="double">
          0.2
        </Constant>        
      </Apply>
    </DerivedField>
    <DerivedField name="FMUE" optype="continuous" dataType="double">
      <Apply function="sum">
        <FieldRef field="FMUE_1"/>
        <FieldRef field="FMUE_2"/>
        <FieldRef field="FMUE_3"/>
        <FieldRef field="FMUE_4"/>
        <FieldRef field="FMUE_5"/>
        <FieldRef field="FMUE_6"/>
        <FieldRef field="FMUE_7"/>
        <FieldRef field="FMUE_8"/>
        <FieldRef field="FMUE_9"/>
        <FieldRef field="FMUE_10"/>
        <FieldRef field="FMUE_11"/>
        <FieldRef field="FMUE_12"/>
        <FieldRef field="FMUE_13"/>
        <FieldRef field="FMUE_14"/>
        <FieldRef field="FMUE_15"/>
        <FieldRef field="FMUE_16"/>
        <FieldRef field="FMUE_17"/>
        <FieldRef field="FMUE_18"/>
        <FieldRef field="FMUE_19"/>
        <FieldRef field="FMUE_20"/>
        <FieldRef field="FMUE_21"/>
        <FieldRef field="FMUE_22"/>
        <FieldRef field="FMUE_23"/>
        <FieldRef field="FMUE_24"/>
        <FieldRef field="FMUE_25"/>
        <FieldRef field="FMUE_26"/>
        <FieldRef field="FMUE_27"/>
        <FieldRef field="FMUE_28"/>
        <FieldRef field="FMUE_29"/>
        <FieldRef field="FMUE_30"/>
        <FieldRef field="FMUE_31"/>
        <FieldRef field="FMUE_32"/>
        <FieldRef field="FMUE_33"/>
      </Apply>
    </DerivedField>
    <DerivedField name="MAS" optype="continuous" dataType="double">
      <Apply function="max">
        <FieldRef field="FINGER_FLEXION_MAS"/>
        <FieldRef field="FINGER_EXTENSION_MAS"/>
        <FieldRef field="WRIST_FLEXION_MAS"/>
        <FieldRef field="WRIST_EXTENSION_MAS"/>
        <FieldRef field="ELBOW_FLEXION_MAS"/>
        <FieldRef field="ELBOW_EXTENSION_MAS"/>
        <FieldRef field="SHOULDER_ANTEVERSION_MAS"/>
        <FieldRef field="SHOULDER_RETROVERSION_MAS"/>
        <FieldRef field="SHOULDER_ABDUCTION_MAS"/>
        <FieldRef field="SHOULDER_ADDUCTION_MAS"/>
        <FieldRef field="SHOULDER_INNER_ROTATION"/>
        <FieldRef field="SHOULDER_OUTER_ROTATION"/>
      </Apply>
    </DerivedField>
    <DerivedField name="HADS_D" optype="continuous" dataType="double">
      <Apply function="sum">
        <FieldRef field="HADS_D_1"/>
        <FieldRef field="HADS_D_2"/>
        <FieldRef field="HADS_D_3"/>
        <FieldRef field="HADS_D_4"/>
        <FieldRef field="HADS_D_5"/>
        <FieldRef field="HADS_D_6"/>
        <FieldRef field="HADS_D_7"/>
      </Apply>
    </DerivedField>
  </TransformationDictionary>
  <TreeModel functionName="classification">
    <MiningSchema>
      <MiningField name="PROXIMAL" missingValueReplacement="0"/>
      <MiningField name="DISTAL" missingValueReplacement="0"/>
      <MiningField name="FMUE" missingValueReplacement="22"/>
      <MiningField name="MAS" missingValueReplacement="0"/>
      <MiningField name="HADS_D" missingValueReplacement="0"/>
      <MiningField name="CORSI" missingValueReplacement="3"/>
      <MiningField name="VAS" missingValueReplacement="0"/>
      <MiningField name="EPILEPSY" missingValueReplacement="1"/>
      <MiningField name="EXPERIMENTAL" missingValueReplacement="0"/>
      <MiningField name="class" usageType="predicted"/>
    </MiningSchema>
    <Node id="0" recordCount="2000">
      <True/>
      <ScoreDistribution value="TIMES" recordCount="166"/>
      <ScoreDistribution value="ECOSS" recordCount="369"/>
      <ScoreDistribution value="AVANCER" recordCount="465"/>
      <ScoreDistribution value="CIMT" recordCount="692"/>
      <ScoreDistribution value="PROXIMAL_ES" recordCount="286"/>
      <ScoreDistribution value="DISTAL_ES" recordCount="922"/>
      <ScoreDistribution value="MIRROR" recordCount="142"/>
      <ScoreDistribution value="PSYCHOTHERAPY" recordCount="28"/>
      <Node id="1" recordCount="1000">
        <SimplePredicate field="EXPERIMENTAL" operator="equal" value="1"/>
        <ScoreDistribution value="TIMES" recordCount="166"/>
        <ScoreDistribution value="ECOSS" recordCount="369"/>
        <ScoreDistribution value="AVANCER" recordCount="465"/>
        <ScoreDistribution value="CIMT" recordCount="346"/>
        <ScoreDistribution value="PROXIMAL_ES" recordCount="143"/>
        <ScoreDistribution value="DISTAL_ES" recordCount="461"/>
        <ScoreDistribution value="MIRROR" recordCount="71"/>
        <ScoreDistribution value="PSYCHOTHERAPY" recordCount="14"/>
      </Node>
      <Node id="2" recordCount="1000">
        <SimplePredicate field="EXPERIMENTAL" operator="equal" value="0"/>
        <ScoreDistribution value="TIMES" recordCount="0"/>
        <ScoreDistribution value="ECOSS" recordCount="0"/>
        <ScoreDistribution value="AVANCER" recordCount="0"/>
        <ScoreDistribution value="CIMT" recordCount="346"/>
        <ScoreDistribution value="PROXIMAL_ES" recordCount="143"/>
        <ScoreDistribution value="DISTAL_ES" recordCount="461"/>
        <ScoreDistribution value="MIRROR" recordCount="71"/>
        <ScoreDistribution value="PSYCHOTHERAPY" recordCount="14"/>
      </Node>
    </Node>
  </TreeModel>
</PMML>

I wanted to test this by inputting a patient with 1's everywhere and verifying that the patient ends up in node 1, with probabilities 0.166 for TIMES, 0.369 for ECOSS, etc... with probabilities not necessarily adding up to 1 (the treatments are not mutually exclusive).

I saved the document into MotorTree.pmml and then in python ran

from pypmml import Model
clf = Model.fromFile("MotorTree.pmml")
cols = ['FMUE_%d'%i for i in range(1,33+1)]\
+['FINGER_FLEXION_MAS', 'FINGER_EXTENSION_MAS','WRIST_FLEXION_MAS','WRIST_EXTENSION_MAS','ELBOW_FLEXION_MAS',\
  'ELBOW_EXTENSION_MAS', 'SHOULDER_ANTEVERSION_MAS','SHOULDER_RETROVERSION_MAS', 'SHOULDER_ABDUCTION_MAS',\
  'SHOULDER_ADDUCTION_MAS', 'SHOULDER_INNER_ROTATION_MAS','SHOULDER_OUTER_ROTATION_MAS']\
+['HADS_D_%d'%i for i in range(1,7+1)]\
+['VAS','CORSI','EXPERIMENTAL','EPILEPSY']
clf.predict(pd.DataFrame(index=[0], columns=cols, data=0))

which returned the dataframe

node_id     predicted_class     probability     probability_AVANCER     probability_CIMT    probability_DISTAL_ES   probability_ECOSS   probability_MIRROR  probability_PROXIMAL_ES     probability_PSYCHOTHERAPY   probability_TIMES
0   2   None    NaN     0.0     0.3343  0.445411    0.0     0.068599    0.138164    0.013527    0.0

It seems there was an automatic scaling to make the probabilities add up to 1, but I don't want this. How can I get around this?

If you have gotten all the way to the bottom of this, a hearty thank you, and an even heartier one if you can provide an answer!

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.