# How does one sample all images uniformly when the data set is organized in hierarchies?

I have a data set with $$C=64$$ classes and $$N_c = 600$$ images and total images $$N = \sum_c N_c$$. Each class has a separate folder for each class. I want to be able to sample images as if they were in a union of all folders (i.e. any image has the same probability no matter where it is located and even if classes are unbalanced). The issue is that I don't want to actually flatten the data set into a union as that duplicates the data and higher storage costs. So I want to sample such that if I were to sample a batch e.g. $$B=1024$$ then every image has the same chance of occurring. I think for that I need that the probability of an image to be equal to:

$$Pr[X = x] = \frac{1}{N}$$

I think that is what my problem boils down to if I am correct (correct me if that is the wrong modeling please!)

So for that to happen I think what I need to do is first sample a class with probability equal $$Pr[C=c] = \frac{N_c C}{N}$$ and sampling an image given that class uniformly i.e. with probability $$Pr[X=x \mid C = c] = \frac{1}{N_c}$$

This is what I get using the marginalization rule for probs:

$$Pr[X=x] = \sum_c p(c) p(x \mid c) = \frac{1}{N}$$

if you plug in the numbers.

But that seems weird, I would have thought that it's sufficient to simply sample any class weighted by the number of images it has and then sample a image uniformly (i.e. just sample uniformly twice $$B$$ times to get your batch).

But formal calculations seem to give me something else, what is my mistake?

probability sampling image from class $$A$$ sampling from whole set is $$P(class\_A)=size(A)/size(whole\_set)$$ but in second sampling/drawing $$P(class\_A)=\frac{size(A)-1}{size(whole\_set)-1}$$ because I presume you are sampling without replacement, of course after picking class/subset you can sample uniformly from this smaller set