A basic multinomial logistic regression model doesn't do great (~20%) in terms of test classification error for my problem, so a thought I had right away was to apply the adaboost to lower that. Initially, I got negative weights on the subsequent classifiers, but I figured out this was because the original adaboost algorithm is meant for binary classification. This paper: https://web.stanford.edu/~hastie/Papers/samme.pdf extends adboost to a multi-class response using a multi-class exponential loss function. The new algorithm ends up being very similar to the original: all we do is add a constant log(K-1) to the weight for the current classifier (K=#of classes).
Anyway, after implementing this the weights all end up positive but the test and training classification error actually increases as the number of classifiers, M, increases which seems bizarre. My guess is that multinomial logistic regression doesn't count as a "weak learner" (if true, why?). Is this something that happens with non-weak learners or could it just be a bug in my code? Or something else?
I moved on to trying something else to lower classification error, so this is more of an open ended curiosity question.