I'm new to ensemble learning and am using three classifiers to identify anomalies using majority voting. I plotted the ROC of each classifier and then created an ensemble by varying the cutoff of each classifier. The plot of the TPR vs 1-FPR is a scatter plot instead of lying on one curve. Can anyone explain please? Thanks
From your description of the setup, I'm assuming you have 3 classifiers, each producing a continuous value output (e.g. 0.1 to 0.9) and varying the cutoff to convert the continuous value into a binary true/false output. If 2 of your classifiers state true, the output is true, otherwise the final output of the ensemble is false.
To plot an ROC curve you have to draw a line graph starting from lowest cutoff (100% false positive rate) to highest cutoff and connect all the points obtained throughout this iteration.
But since there are 3 classifiers, you need to plot 3 ROC curves separately; although you could plot them on the same graph.
Since the key requirement of an ensemble is that classifiers should have low correlation among themselves, applying a uniform cutoff on all 3 and plotting a single ROC curve will not be helpful to reveal how your ensemble will perform for different cutoffs.