Is it wise to use multiple binary models as opposed to a multinomial model? Let's say that you have a classification problem where the dependent variable has MANY levels (say 20) and you CANNOT transform the target (i.e. no clustering, combining of levels, etc.). Is it a good idea to fit a binary model for each level (i.e. Y = 1 if specific level versus Y= 0 otherwise) as opposed to a model where you are trying to predict all of the level of Ys simultaneously?
I know that when you are using logistic regression, the multinomial approach is more efficient, but will this hold when employing machine learning algorithms? So far the best model was a boosted tree. Any thoughts on this?  
For the choice of the predicted value, I was thinking that I would just predict whichever category corresponded to the highest probability.
 A: The approach using binary classifiers against multiclass data instead of methods designed especially for such data is commonly used in machine learning and implemented in multiple software packages. Clear description of such procedures can be found in Python's scikit-learn library documentation:

One-Vs-The-Rest
This strategy, also known as one-vs-all, is implemented in
  OneVsRestClassifier. The strategy consists in fitting one classifier
  per class. For each classifier, the class is fitted against all the
  other classes.
(...)
One-Vs-One
OneVsOneClassifier constructs one classifier per pair of classes. At
  prediction time, the class which received the most votes is selected.
  In the event of a tie (among two classes with an equal number of
  votes), it selects the class with the highest aggregate classification
  confidence by summing over the pair-wise classification confidence
  levels computed by the underlying binary classifiers.

The approach that you mention

For the choice of the predicted value, I was thinking that I would
  just predict whichever category corresponded to the highest
  probability.

is a description of one-vs-rest classifier (see Wikipedia article on multiclass classification). You can find more detailed description of these approaches in most of the introductory machine learning handbooks.
