Let as assume that we have a binary classification problem. We also have several classifiers. Instead of assigning a vector to a class (0 or 1) each classifier returns a probability that a given vector belongs to class 1. It means that for each input vector, that has to be classified we get a vector of real number between 0 and 1. For example:
(0.81, 0.67, 0.43, 0.99, 0.53)
where the number of components (probabilities) is equal to the number of classifiers. Now we want to "combine" these "weak" classifiers to get on "strong" classifier. In other words we need to find a way to map a given vector of probabilities into one number (probability).
So, my questions is: What is the "correct" way to do it? Of course I can train another classifier that uses the vector of probabilities and returns one probability. In other words we can find out how to combine the "weak" probabilities in an empirical way. However, I assume, that we can use the fact that the components of the vector are not just "some numbers" (or features) they are probabilities, they are already predictions and, as a consequence, they have to be combined in a corresponding appropriate way.
In comments it has been proposed to average the "weak" probabilities. But what if it is possible to estimate quality (power) of each "weak" classifier (and it should be possible), doesn't it make sense to suppress "bad" classifier (for example by using their predictions (probabilities) with smaller weights or by ignoring them completely)? Does it makes sense to use just one (the best) weak classifier? Does it make sense to check correlation between the weak classifiers. For example what should we do if two "weak" classifiers always give the same result. Shouldn't we through one of them out as not having any additional value?