How to construct a combined binary classifier? I have two binary classifiers, a logistic regression classifier (returning probabilities) and a decision tree classifier (returning 1 and 0).
I was wondering if it is possible to combine the prediction of both, but I am not sure what is the strategy. Should I somehow try to find the probabilities from the decision tree classifier and multiply them between them or maybe get the mean predictions divided by two?
Or should I perhaps change the logistic regression probability outputs to classifications (0,1) and try to sort it out somehow this way? I think the latter sounds wrong to me. I assume we would like to use the probabilities, i just don't know how to get the probabilities out of the decision tree classifier :/
By the way I am running this in R using the glm and tree functions repspectively
 A: When you combine models it's called an Ensemble model. Ensembles use voting, weighting, and or averaging, and there are different algorithms.
(Averaging) Most decision tree functions will give you the probability.  You could just average the two.
(Boosting) Lets say your logistic regression model performs better with the bottom 50% of values of a continuous independent variable, and your decision tree performs better with the top 50% of values of that continuous variable. Your ensemble could choose stronger model for that scenario.
(Voting) As a different example, if you had three models, you could use simple voting, and not use an analysis of the training data (which was done above).
In response to the comment, I've addressed the "how are models combined" with three examples, and provided the term "ensemble."  The "should I" is a different question, which will depend.
A: Make your classification tree algorithm output probabilities, not hard 0-1 classifications. See here on the rationale, quite independently of your ensembling situation.
Then you have two probabilistic classifiers. Simply combine the probabilistic predictions within each class by averaging, possibly using weights.
