Return to Answer

By using stacking you are including an extra step that the machine needs to learn. You need to spend data on this which means that it is not only able to improve the situation but can also make it worse.

• Did you still train the individual base learners with the same data before the step with the meta learner? Possibly your algorithm allocates less data for this learning step because it also needs to train and validate the stacking.

• And how good is this last training step? If the training is done badly then this stacking step can ruin the otherwise good models. Anyways, with two models you are not gonna improve much. How are you gonna stack them when one says the class is 1 and the other says the class is 0? A man with a watch knows what time it is. A man with two watches is never sure. This is why ships used to take three clocks along and not two.

Besides the above considerations, it might be that the different classifiers can have a difficult correlation for classes such that a combination of the classifier works out badly.

Below is an example where the correlation between the two classifiers is negative for the one class but positive for the other. E.g. if the class 2 is the positive case then the two classifiers often agree on the positive case class, but disagree on the negative case classes.

In the right image, we see what happens when we would use an average of the two classifiers. For the high TPR rates the FPR is worse.

Back to the left image, it is this part on the lower left of the 'class 2' that is difficult to classify when we combine the two classifiers according to some weighted mean. We have plotted three boundary lines that relate to a False Positive Rate of 0.8. The horizontal and vertical boundary lines (using only a single of the classifiers) do better than the diagonal line (relating to a mean of the classifiers).

I have tried to replicate the above images for your case. It is slightly present as well. We can ascribe this to the situation where you have only 32 data points. This makes it easier that you randomly get such peculiar situation.

Something that I can not reproduce is the situation where you have very low initial TPR. The ROC curve for the individual base learners rapidly increases to above 0.5 without an increase in FPR. We do not see this in the stacked model. It is unclear what sort of combination of the two base learners is making such error and it seems like this might be a coding error. When I simply use the average of the two classifiers then I do not get this discrepancy.

By using stacking you are including an extra step that the machine needs to learn. You need to spend data on this which means that it is not only able to improve the situation but can also make it worse.

• Did you still train the individual base learners with the same data before the step with the meta learner? Possibly your algorithm allocates less data for this learning step because it also needs to train and validate the stacking.

• And how good is this last training step? If the training is done badly then this stacking step can ruin the otherwise good models. Anyways, with two models you are not gonna improve much. How are you gonna stack them when one says the class is 1 and the other says the class is 0? A man with a watch knows what time it is. A man with two watches is never sure. This is why ships used to take three clocks along and not two.

Besides the above considerations, it might be that the different classifiers can have a difficult correlation for classes such that a combination of the classifier works out badly.

Below is an example where the correlation between the two classifiers is negative for the one class but positive for the other. E.g. if the class 2 is the positive case then the two classifiers often agree on the positive case class, but disagree on the negative case classes.

In the right image, we see what happens when we would use an average of the two classifiers. For the high TPR rates the FPR is worse.

Back to the left image, it is this part on the lower left of the 'class 2' that is difficult to classify when we combine the two classifiers according to some weighted mean. We have plotted three boundary lines that relate to a False Positive Rate of 0.8. The horizontal and vertical boundary lines (using only a single of the classifiers) do better than the diagonal line (relating to a mean of the classifiers).

I have tried to replicate the above images for your case. It is slightly present as well. We can ascribe this to the situation where you have only 32 data points. This makes it easier that you randomly get such peculiar situation.

Something that I can not reproduce is the situation where you have very low initial TPR. The ROC curve for the individual base learners rapidly increases to above 0.5 without an increase in FPR. We do not see this in the stacked model. It is unclear what sort of combination of the two base learners is making such error and it seems like this might be a coding error. When I simply use the average of the two classifiers then I do not get this discrepancy.

I guess that your data in 'dataset.csv' relates to the logistic regression. Here the stacked regression is not doing so bad. The only point where it is worse than the two models together is around the point FPR = 0.7 and TPR = 0.85.

If you provide the dataset for the other cases, then we can see if it corresponds to the example above. Plots of the scores for the two classifier scores will give some insight.

By using stacking you are including an extra step that the machine needs to learn. You need to spend data on this which means that it is not only able to improve the situation but can also make it worse.

• Did you still train the individual base learners with the same data before the step with the meta learner? Possibly your algorithm allocates less data for this learning step because it also needs to train and validate the stacking.

• And how good is this last training step? If the training is done badly then this stacking step can ruin the otherwise good models. Anyways, with two models you are not gonna improve much. How are you gonna stack them when one says the class is 1 and the other says the class is 0? A man with a watch knows what time it is. A man with two watches is never sure. This is why ships used to take three clocks along and not two.

Besides the above considerations, it might be that the different classifiers can have a difficult correlation for classes such that a combination of the classifier works out badly.

Below is an example where the correlation between the two classifiers is negative for the one class but positive for the other. E.g. if the class 2 is the positive case then the two classifiers often agree on the positive case class, but disagree on the negative case classes.

In the right image, we see what happens when we would use an average of the two classifiers. For the high TPR rates the FPR is worse.

Back to the left image, it is this part on the lower left of the 'class 2' that is difficult to classify when we combine the two classifiers according to some weighted mean. We have plotted three boundary lines that relate to a False Positive Rate of 0.8. The horizontal and vertical boundary lines (using only a single of the classifiers) do better than the diagonal line (relating to a mean of the classifiers).

I have tried to replicate the above images for your case. It is slightly present as well. We can ascribe this to the situation where you have only 32 data points. This makes it easier that you randomly get such peculiar situation.

Something that I can not reproduce is the situation where you have

By using stacking you are including an extra step that the machine needs to learn. You need to spend data on this which means that it is not only able to improve the situation but can also make it worse.

• Did you still train the individual base learners with the same data before the step with the meta learner? Possibly your algorithm allocates less data for this learning step because it also needs to train and validate the stacking.

• And how good is this last training step? If the training is done badly then this stacking step can ruin the otherwise good models. Anyways, with two models you are not gonna improve much. How are you gonna stack them when one says the class is 1 and the other says the class is 0? A man with a watch knows what time it is. A man with two watches is never sure. This is why ships used to take three clocks along and not two.

Besides the above considerations, it might be that the different classifiers can have a difficult correlation for classes such that a combination of the classifier works out badly.

Below is an example where the correlation between the two classifiers is negative for the one class but positive for the other. E.g. if the class 2 is the positive case then the two classifiers often agree on the positive case class, but disagree on the negative case classes.

In the right image, we see what happens when we would use an average of the two classifiers. For the high TPR rates the FPR is worse.

Back to the left image, it is this part on the lower left of the 'class 2' that is difficult to classify when we combine the two classifiers according to some weighted mean. We have plotted three boundary lines that relate to a False Positive Rate of 0.8. The horizontal and vertical boundary lines (using only a single of the classifiers) do better than the diagonal line (relating to a mean of the classifiers).

I have tried to replicate the above images for your case. It is slightly present as well. We can ascribe this to the situation where you have only 32 data points. This makes it easier that you randomly get such peculiar situation.

Something that I can not reproduce is the situation where you have very low initial TPR. The ROC curve for the individual base learners rapidly increases to above 0.5 without an increase in FPR. We do not see this in the stacked model. It is unclear what sort of combination of the two base learners is making such error and it seems like this might be a coding error. When I simply use the average of the two classifiers then I do not get this discrepancy.

By using stacking you are including an extra step that the machine needs to learn. You need to spend data on this which means that it is not only able to improve the situation but can also make it worse.

• Did you still train the individual base learners with the same data before the step with the meta learner? Possibly your algorithm allocates less data for this learning step because it also needs to train and validate the stacking.

• And how good is this last training step? If the training is done badly then this stacking step can ruin the otherwise good models. Anyways, with two models you are not gonna improve much. How are you gonna stack them when one says the class is 1 and the other says the class is 0? A man with a watch knows what time it is. A man with two watches is never sure. This is why ships used to take three clocks along and not two.

By using stacking you are including an extra step that the machine needs to learn. You need to spend data on this which means that it is not only able to improve the situation but can also make it worse.

• Did you still train the individual base learners with the same data before the step with the meta learner? Possibly your algorithm allocates less data for this learning step because it also needs to train and validate the stacking.

• And how good is this last training step? If the training is done badly then this stacking step can ruin the otherwise good models. Anyways, with two models you are not gonna improve much. How are you gonna stack them when one says the class is 1 and the other says the class is 0? A man with a watch knows what time it is. A man with two watches is never sure. This is why ships used to take three clocks along and not two.

Besides the above considerations, it might be that the different classifiers can have a difficult correlation for classes such that a combination of the classifier works out badly.

Below is an example where the correlation between the two classifiers is negative for the one class but positive for the other. E.g. if the class 2 is the positive case then the two classifiers often agree on the positive case class, but disagree on the negative case classes.

In the right image, we see what happens when we would use an average of the two classifiers. For the high TPR rates the FPR is worse.

Back to the left image, it is this part on the lower left of the 'class 2' that is difficult to classify when we combine the two classifiers according to some weighted mean. We have plotted three boundary lines that relate to a False Positive Rate of 0.8. The horizontal and vertical boundary lines (using only a single of the classifiers) do better than the diagonal line (relating to a mean of the classifiers).

I have tried to replicate the above images for your case. It is slightly present as well. We can ascribe this to the situation where you have only 32 data points. This makes it easier that you randomly get such peculiar situation.

Something that I can not reproduce is the situation where you have