How does an ensemble classifier merge the predictions of its constituent classifiers? I'm having difficulty finding a clear description. In some code examples I've found, the ensemble just averages the predictions, but I don't see how this could possible make a "better" overall accuracy.
Consider the following case. An ensemble classifier is composed of 10 classifiers. One classifier is has an accuracy of 100% of the time in data subset X, and 0% all other times. All other classifiers have an accuracy of 0% in data subset X, and 100% all other times.
Using an averaging formula, where classifier accuracy is ignored, the ensemble classifier would have, at best, 50% accuracy. Is this correct, or am I missing something? How can taking the average prediction from N potentially clueless classifiers possibly create a better prediction then a single classifier that's an expert in a specific domain?