I am currently working on an image-classification problem with an unbalanced dataset, namely one with very different number of training examples within each class. I am trying to understand why such unbalance could incur a problem. (I am using a CNN classifier.)

To start with, I thought about an example of medical testing.

  • Let's say, for a disease A, only a small proportion of the world's population has it. Learning to classify whether a person has disease A is an imbalanced-dataset problem, because the number of patients $<<$ the number of healthy individuals.
  • There is a test to detect disease A, but this test does not always give the correct classification. In other words, the test can be positive when someone does not have disease A; it can be negative when someone does have disease A.
  • Question: given a positive test result, what is the probability that the individual actually has disease A?
  • Solution: $p(\text{has A} | \text{pos. result})=\frac{p(\text{pos. result} | \text{has A})p(\text{has A})}{p(\text{pos. result})}$
  • What I think is interesting in this solution:
    1. Everything here can be clearly illustrated on the same Venn diagram.
    2. $p(\text{has A} | \text{pos. result}) \propto p(\text{has A})$, which means that the posterior depends on prior.

But when I think about the problem that I am working on, I am suddenly confused about these two interesting conclusions. In the problem that I am working on, the new information is not a straightforward test result but an image.

Expressed using Bayes' theorem, the problem is $p(\text{has Cancer} | \text{Scan})=\frac{p(\text{Scan} | \text{has Cancer})p(\text{has Cancer})}{p(\text{Scan})}$.

What I need help with:

  1. How to interpret $p(\text{Scan})$ if each scan is just an array of pixels?
  2. The key difference between my example and the medical-testing example is that: in the medical-testing example, the test result can be incorrect; however, in my cancer-scan example, all the information on the presence of cancer is available in the image (ideally, the scan can't be "incorrect"). Where is this difference reflected in Bayes' thereom? Can I just ignore the prior in this case (by, for example, upsampling the number of cancer cases)?

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.