Decision Trees disadvantage : Inability to examine more than one attribute at a time I'm reading up on decision trees and in Negnevitsky (2002), one significant limitation on the decision tree is their inability to examine more than one variable at a time. 
I've been trying to understand this drawback and hope that someone can help explain it in simple terms. Why is this considered as a drawback? 
Looking at it from the speed angle, by examining one attribute at a time would make the training/testing process faster, isn't it?
 A: For selecting each of the splits in most tree algorithms, only one of explanatory variables can be selected. Hence, if decisions need to depend on several variables, this (a) may miss relevant splits and (b) may need several splits to capture a simple decision.
The classical examples are the chessboard (left panel below) and a linear discrimination rule (right panel below). The binary response is brought out by the red triangles vs. blue circles. The two numeric explanatory variables (or regressors or attributes, etc.) are x1 and x2.

The decision in the left plot can be easily captured by rectangular splits: (x1 > 0.5 & x2 > 0.5) | (x1 < 0.5 & x2 < 0.5) for the red triangles. However, if you search for splits along x1, then you will always have approximately a 50:50 between triangles and circles. Hence many tree algorithms have a harder time finding the correct splits.
The decision on the right can also be easily described: x2 > x1 for the red triangles. However, most decision tree algorithms do not consider such splitting rules and would approximate it by a series of binary splits that would lead to a step function in the plot.
There are decision trees that explicitly try to address these problems, e.g., by considering splits in interactions/products of the variables etc. Another solution would be to consider methods that can capture this better, e.g., forests or boosting etc.
