Implications of using a regression tree for a classification problem

Question

I have a classification problem into yes/no cases. I know that I could use a classification tree that would generate an output of yes or no at each leaf node. However, would it be suitable to use a regression tree in this setting?

My understanding is that

1. For a classification tree splits are picked to minimise entropy: i.e. create the greatest cluster of one class in one node and the greatest cluster of the complement class in the other node.

2. For a regression tree splits are picked to minimise mean squared error. In the case of using a regression tree on yes/no data, the mean squared error at each node would be given by the average of summation terms of the form $1 - m$ and $0 - m$ where $m$ is the proportion of yes's on that node.

I have three questions based on this:

1. Are the two methods of splitting at each node approximately equivalent?

2. The result of using a regression tree on classification data would result in predictions at each leaf node equal to the proportion of times yes was observed in that leaf node. If we convert this proportion into a yes/no classification using an appropriate threshold, would we recover some semblance of a classification tree?

3. If I were interested in the probability of a case falling into the yes category, can I use regression trees for this?

Here is an example on the ionosphere dataset, with accompanying Matlab code:

load ionosphere
ctree = fitctree(X,Y)
ctree
view(ctree, 'mode', 'graph')

which gives

and

Yp = cellfun(@(C) C == 'g', Y);
rtree =fitrtree(X,Yp)
view(rtree, 'mode', 'graph')

which gives

The only difference I can see is one split at a node, circled in red in the regression tree.

Answer 1

First off, it sometimes makes sense to use regression trees in a classification problem. I found it useful when I had examples labeled as yes/no/maybe, and I assigned 0.5 to the maybe cases. If you only have a binary variable, you're better off using classification trees. To answer your questions:

1. They are not equivalent, but as your example demonstrates they sometimes give similar split rules.
2. In regression trees, the predicted value follows the regression model used at the leaf nodes. The model may be using the feature-space value X at the leaf to determine the prediction y, for example a linear fit. In that case, in the example below, with even proportion of positive and negative examples (blue circles) but with uneven spread in the X values, the prediction y (red line) depends on the value of X. This is very different from the classification value, which is equal to the proportion of examples.
3. You can, but only if you have a good reason for it, like non-binary labels.