A question on no. of training examples and decision trees

Question

I have a set of around 200,000 training instances. Each training instance consists of an attribute called $duration$, a discrete integer type and a time series of floating-point values, in form of a vector of length=$duration$. So if each training instance is to be considered a vector, it is of a length $duration+1$.

The problem is is to build a classifier and while I was working on the problem, I tried the following approach:

Because each training vector had a different length equal to the duration attribute, I decided to build different classifiers for different durations. In this way, I could do away with the problem of comparing 2 time series of different lengths.

Now I had to divide the data into groups where each member would have the same duration. This led to segregating the data into 20 groups (for different durations) with 10000 instances each. Several have questioned me saying that 10,000 might not be enough to build a model at all. I have tried to convince myself that such a 10,000 may be enough by the following reasoning:

Take for example a decision tree. It works by deciding on a particular attribute and building recursively, sub-trees as its children. When the subtrees are created, they are done so by using lesser number of training instances than the number used for training its parent node as the the training instances used to train the subtree is a subset of instances in the training set which have a specific value of the attribute on the parent node as per the branch. In this manner, as a decision tree grows down, the no. of training instances it uses to create a sub-tree reduces, but it doesnt mean that the subtrees down the tree by any means are incorrect or unjustified.

Questions:

When I divided my data into groups and built for models for each group, didnt I do a similar thing (and implicitly gave a decision tree like structure to the classifier)? Are my colleagues right in questioning my approach?

Also note that duration is an attribute which contributes heavily towards learning the target function. By dividing into groups and building a classifier for each group, am I not able to capture the effect of duration on the target? Are there better ways to achieve results in this case?

Answer 1

You manually impose a decision-tree-like structure by training 20 different classifiers, which is legitimate. However, I would argue that the decision tree would pick up duration as the root node, if the length of the time series is actually the most important factor in class discrimination. So you may be unnecessarily intervening in the training here.

Also, I assume the length of each instance is finalised before you classify them , since otherwise you wouldn't be able to choose which of the 20 classifiers to feed a new instance into.

An alternative here is to use descriptive statistics of the time series (the dead simple ones being: max, min, median, mean, variance, length, etc.) as features to train a single (central) classifier. The downside is, this will lose information unless you are sure about and careful to extract the specific characteristics of the time series you are interested in for your classifier.

Some notes:

10,000 observations per classifier may be plenty or too little, depending on the dimensions of your probability space. As the curse of dimensionality dictates: with a fixed number of training samples, the predictive power of your classifier reduces as the dimensionality of the probability space increases. In other words, an enormous amount of training data may be required to ensure that there are several samples with each combination of values.

You note that as a decision tree grows down, the no. of training instances it uses to create a sub-tree reduces which is completely true. If you do not have sufficient training data, you may run out of training instances before you hit a leaf node.