Multiplying SVM-derived probabilities with different levels of support

Question

Let's assume there is a function of interest:

y = abc

...where a, b, and c are the probabilities of independent events. I've created classification algorithms (specifically, SVMs) to estimate those three probabilities (with isotonic regression performed on the raw probability outputs for scaling purposes).

My concern is that multiplying the probabilities may still be invalid due to (1) unbalanced classes for all three classifiers, and (2) different proportions of imbalance between the classifiers:

There are 4000 instances in which clf_a = 0.
There are 1000 instances in which clf_a = 1.

There are 3500 instances in which clf_b = 0.
There are 1500 instances in which clf_b = 1.

There are 1500 instances in which clf_c = 0.
There are 3500 instances in which clf_c = 1.

Because classifier_c has only 1500 instances supporting class 0 while the other two classifiers have support of 3500-4000 each (and vice versa for class 1), the resulting accuracies (and other summary statistics) for the classifiers are rather different. For example:

clf_a recall for 0: 0.96
clf_a recall for 1: 0.84

clf_b recall for 0: 0.95
clf_b recall for 1: 0.87

clf_c recall for 0: 0.81
clf_c recall for 1: 0.98

Is my concern about the differing proportions of support for the three classifiers a valid one, even though I'm scaling the initial probability outputs from the classifiers (again, via isotonic regression, although I see similar results when using Platt scaling)? If so, are there steps I can take to resolve the issue, such that I can eventually multiply a, b, and c? Or are there more basic flaws with this approach?

I cannot acquire additional data. I have yet to try oversampling and undersampling. I've tried applying a cost matrix for class weights to the SVMs, though, and summary statistics are largely unchanged. I have also tried other classifiers; the situation is the same for all of them, so this isn't an issue specific to SVMs.

I should emphasize that while I want to see robust, powerful classifiers, my primary goal is to estimate probabilities for which subsequent multiplication is mathematically valid.

Thank you in advance for any thoughts, comments, or ideas you might have.

Answer 1

After speaking with a distinguished researcher in the field of machine learning, I came away with the following:

1. If the individual classifiers are working with extremely unbalanced data, the resulting probability estimates are probably suspect regardless of what one does with them.
2. If some technique is used to account for these unbalanced data, subsequent multiplication of the individual probabilities is valid so long as I'm absolutely sure of each event's independence.
3. Some other approaches to resolve my data imbalance were offered, most of which are covered or at least touched on in a recent survey paper (He et al. 2009).

I found that random oversampling partially resolved my issues with the classifiers, and that targeted oversampling did even better. The general sense I got from the researcher, however, is that multiplying the probabilities was perfectly fine if the probability estimates themselves were good estimates, but that those probabilities were suspect when dealing with unbalanced data sets, and especially when different classes from different classifiers exhibited different forms of imbalance.

The full citation of He's survey paper is given here:

He, H., & Garcia, E. A. (2009). Learning from imbalanced data. IEEE Transactions on knowledge and data engineering, 21(9), 1263-1284.