1
$\begingroup$

I am currently working on an image-classification problem with an unbalanced dataset, namely one with very different number of training examples within each class. I am trying to understand why such unbalance could incur a problem. (I am using a CNN classifier.)

To start with, I thought about an example of medical testing.

  • Let's say, for a disease A, only a small proportion of the world's population has it. Learning to classify whether a person has disease A is an imbalanced-dataset problem, because the number of patients $<<$ the number of healthy individuals.
  • There is a test to detect disease A, but this test does not always give the correct classification. In other words, the test can be positive when someone does not have disease A; it can be negative when someone does have disease A.
  • Question: given a positive test result, what is the probability that the individual actually has disease A?
  • Solution: $p(\text{has A} | \text{pos. result})=\frac{p(\text{pos. result} | \text{has A})p(\text{has A})}{p(\text{pos. result})}$
  • What I think is interesting in this solution:
    1. Everything here can be clearly illustrated on the same Venn diagram.
    2. $p(\text{has A} | \text{pos. result}) \propto p(\text{has A})$, which means that the posterior depends on prior.

But when I think about the problem that I am working on, I am suddenly confused about these two interesting conclusions. In the problem that I am working on, the new information is not a straightforward test result but an image.

Expressed using Bayes' theorem, the problem is $p(\text{has Cancer} | \text{Scan})=\frac{p(\text{Scan} | \text{has Cancer})p(\text{has Cancer})}{p(\text{Scan})}$.


What I need help with:

  1. How to interpret $p(\text{Scan})$ if each scan is just an array of pixels?
  2. The key difference between my example and the medical-testing example is that: in the medical-testing example, the test result can be incorrect; however, in my cancer-scan example, all the information on the presence of cancer is available in the image (ideally, the scan can't be "incorrect"). Where is this difference reflected in Bayes' thereom? Can I just ignore the prior in this case (by, for example, upsampling the number of cancer cases)?
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.