Does anyone know of a good reference that list the reasons for transforming multiple class classification problem into a set of binary sub-problems?
In response to comment: One reason to transform a multiple class classification problem into a set of binary sub-problems are that a binary network is usually simpler and therefore better suited to fit small data sets (Less prone to over fitting).
Furnkranz argues that a set of simpler binary classifiers can also provide better performance than one complex multiple class classifier because it is often easier to learn how to distinguish between two classes, than it is to learn how to distinguish between multiple classes. Simpler models reduce training time also.
Milgram shows that a set of binary SVM can preform better than a multi-class MLP. Though this might be due to choice of classifier rather than binarization.
If we focus on a MLP classifier, training a set of binary one-against-all MLPs and combining using softmax function would be very much equal the the multi-class MLP however this would allow us to use different hyper-paramater settings for each class.