# How can you use a decision tree classifier on nested data?

I have a data model with natural one-to-many relationships.

User:
Job1:
StartDate: May, 2010
EndDate: Jun, 2014
RoleName: SeniorDataScientist
EmploymentType: FT
Job2:
StartDate: May, 2009
EndDate: May, 2010
Role: JuniorDataScientist
EmploymentType: PT


I am trying to predict which people will be a good fit for a particular job.

I've built decision trees where I take the data in the work history and concatenate it together into a single vector for training my decision tree --

User:
EmploymentLength: 5
RoleNames: DataScientist,SeniorDataScientist
EmploymentType: PT,FT


(The text-based features get processed by a bag of words model)

But doing this loses the contextual information available in the relational model --> that is, I lose the information telling me that the senior position is full time and associated with a 4 year timespan while the junior position is only 1 year and is associated with the part time position.

I've simplified my example a little bit. My model actually has a few one-to-many relationships so I'm going to need to do this to other relationships as well.

I'm looking for a general way to process these nested relationships for my classifier (or use some other technique than decision trees to handle data that looks like this). What technique can be used to process this kind of data or predict an outcome with a classifier using nested features like these?

You may want to look into the area known as statistical relational learning; there's a book called Introduction to Statistical Relational Learning edited by Lise Getoor and Ben Taskar.

In particular, there's a family of decision tree-based methods in this area known as relational probability trees; an early paper is Neville, Jensen, Friedland, and Hay, Learning Relational Probability Trees, ACM SIGKDD 2003.

There are also extensions to spatiotemporal data, with extensive applications to meteorology, by Amy McGovern and collaborators; a fairly recent journal paper is McGovern, Troutman, Brown, Williams, and Abernethy, Enhanced spatiotemporal relational probability trees and forests, Data Mining and Knowledge Discovery 2012.

I don't know offhand of publicly available code for these methods, but it's a place to start.

Two popular relational learners are TILDE and FOIL. TILDE is a relational decision tree learner, which comes bundled with ACE.

Another good book in the same domain is by Luc De Raedt, Logical and Relational Learning, Springer, 2008.

Although there is nesting and relations here, they are time-dependent. The best model is one that processes sequences.

Each user is a sequence of jobs. Each job is a vector. Role name and employment type could just be one-hot encoded. The dates could be treated the same, or even better incorporated somehow into the structure of the sequences. You could get arbitrarily creative with how you would like to represent these raw data, but if done right you will lose almost no information. Especially, as a sequence, you retain information on order, which was not present before. Otherwise, you treat an "engineer turned manager" the same as a "manager turned engineer", which is not the case.

Now, you can take advantage of all of the work done in this area (e.g., RNN, LSTM, GRU, RA) as well as many powerful libraries in tow (Keras, TF, Theano).