Hi everybody I've been struggling around with a question about probabilities. I have a background in physics, so I though I was able to handle it but after some days thinking I just give up. Here it is:

I want to classify a large group of directories (thousands). Each of them contain hundreds of images. The images can be for example: cats, dogs, birds, cars, etc. Some directories can contain mostly images of one type, others a mix of them.

I have an algorithm that takes one image and prints out the probabilities that this image belong to a pre-trained set of images (Random Forest). eg. 0.8 * cat, 0.1 * dog, 0.1 * bird, 0.0 * others... Because I cannot train all possible images in the directories, I also have a 'others' type of images ('0' class).

My task is to classify the directories by estimating the probability(ies) that a chosen directory contains images of a given type(s) without to check all images inside, but for example picking up randomly a certain number $n$ of images of a total of $N$ and use their individual result from the algorithm.

Is this a good idea for classifying all my directories? Statistically speaking how can I formulate my problem. Any hint or idea is welcome.

Thanks in advance!

  • $\begingroup$ Your questions needs clarification. Do you mean the probability of all images to be in the same class or set of class? 'Best' does not make sence, you need to give more information on the application and your thoughts on evaluation. What are your result? $\endgroup$ Aug 18 '17 at 7:41
  • $\begingroup$ 1. If there are $m$ classes (presumably mutually exclusive), why does the index on $i$ go from $0$ to $m$ (i.e. $m+1$ possibilities)? 2. You describe images belonging to a class, but ask about directories belonging to a class without explaining how this class attribute of images relates to directories. There's something important missing here. 3. You can't ask for "best" without explaining what you're trying to do "best" at --- what's your measure of goodness? 4. It's not clear what you mean by "confidence" here. It doesn't seem to correspond to the way confidence is used in statistics. $\endgroup$
    – Glen_b
    Aug 18 '17 at 7:44
  • $\begingroup$ thanks for the comments, I did edit my question a little bit. I hope it is clear enough. BR $\endgroup$ Aug 18 '17 at 8:22
  • $\begingroup$ Are you trying to figure out "does this directory contain any images in class $i$?" or are you trying to figure out something else? $\endgroup$
    – Glen_b
    Aug 18 '17 at 8:40
  • $\begingroup$ I'm trying to figure out if a given directory mostly contains images of a given class $i$. eg. A directory $D$ contains mostly images of cats (0.8), dogs (0.1) and others (0.1) without to check the whole content. $\endgroup$ Aug 18 '17 at 8:47

It sounds like there are two separate problems:

  1. predict the class that the image belongs to
  2. predict the class the directory belongs to (which takes as input, the classes of images)

For a sample of directories, with 100% classified images, you could experiment with the value of n images, that gives you a sufficiently accurate result for the directory prediction. You could also randomly sample these directories to better understand the distribution of images, which might also give you an indication of what size n should be.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.