probability a directory belongs to a class given classified images in it

Hi everybody I've been struggling around with a question about probabilities. I have a background in physics, so I though I was able to handle it but after some days thinking I just give up. Here it is:

I want to classify a large group of directories (thousands). Each of them contain hundreds of images. The images can be for example: cats, dogs, birds, cars, etc. Some directories can contain mostly images of one type, others a mix of them.

I have an algorithm that takes one image and prints out the probabilities that this image belong to a pre-trained set of images (Random Forest). eg. 0.8 * cat, 0.1 * dog, 0.1 * bird, 0.0 * others... Because I cannot train all possible images in the directories, I also have a 'others' type of images ('0' class).

My task is to classify the directories by estimating the probability(ies) that a chosen directory contains images of a given type(s) without to check all images inside, but for example picking up randomly a certain number $n$ of images of a total of $N$ and use their individual result from the algorithm.

Is this a good idea for classifying all my directories? Statistically speaking how can I formulate my problem. Any hint or idea is welcome.

• Your questions needs clarification. Do you mean the probability of all images to be in the same class or set of class? 'Best' does not make sence, you need to give more information on the application and your thoughts on evaluation. What are your result? Aug 18 '17 at 7:41
• 1. If there are $m$ classes (presumably mutually exclusive), why does the index on $i$ go from $0$ to $m$ (i.e. $m+1$ possibilities)? 2. You describe images belonging to a class, but ask about directories belonging to a class without explaining how this class attribute of images relates to directories. There's something important missing here. 3. You can't ask for "best" without explaining what you're trying to do "best" at --- what's your measure of goodness? 4. It's not clear what you mean by "confidence" here. It doesn't seem to correspond to the way confidence is used in statistics. Aug 18 '17 at 7:44
• thanks for the comments, I did edit my question a little bit. I hope it is clear enough. BR Aug 18 '17 at 8:22
• Are you trying to figure out "does this directory contain any images in class $i$?" or are you trying to figure out something else? Aug 18 '17 at 8:40
• I'm trying to figure out if a given directory mostly contains images of a given class $i$. eg. A directory $D$ contains mostly images of cats (0.8), dogs (0.1) and others (0.1) without to check the whole content. Aug 18 '17 at 8:47