I have a classification problem into yes/no cases. I know that I could use a classification tree that would generate an output of yes or no at each leaf node. However, would it be suitable to use a regression tree in this setting?
My understanding is that
For a classification tree splits are picked to minimise entropy: i.e. create the greatest cluster of one class in one node and the greatest cluster of the complement class in the other node.
For a regression tree splits are picked to minimise mean squared error. In the case of using a regression tree on yes/no data, the mean squared error at each node would be given by the average of summation terms of the form $1 - m$ and $0 - m$ where $m$ is the proportion of yes's on that node.
I have three questions based on this:
Are the two methods of splitting at each node approximately equivalent?
The result of using a regression tree on classification data would result in predictions at each leaf node equal to the proportion of times yes was observed in that leaf node. If we convert this proportion into a yes/no classification using an appropriate threshold, would we recover some semblance of a classification tree?
If I were interested in the probability of a case falling into the yes category, can I use regression trees for this?
Here is an example on the ionosphere dataset, with accompanying Matlab code:
load ionosphere
ctree = fitctree(X,Y)
ctree
view(ctree, 'mode', 'graph')
and
Yp = cellfun(@(C) C == 'g', Y);
rtree =fitrtree(X,Yp)
view(rtree, 'mode', 'graph')
The only difference I can see is one split at a node, circled in red in the regression tree.