Cross checking AI Selections vs Random Chance

Fifty Three Images

AI Selects 20 images based on various criteria. I Select 20 images based on personal criteria.

In a two group selection of all 53 items what is the likely overlap due to random chance?

• Are you looking for the probability that at least one image is selected twice or do you want to know how many images that are selected twice you can reasonably expect with p=0.95? – mroman Nov 11 '18 at 16:48
• How many images selected twice can I reasonably expect? – Rikk Flohr Nov 12 '18 at 0:49
• Then see my answer. You can build a tree to calculate all the probabilites and then you pick the leaf with the highest probability. – mroman Nov 12 '18 at 10:29

If $$A$$ selects $$n$$ images and $$B$$ selects $$m$$ images out of $$t$$ total images. You can take those first $$n$$ images as a reference frame and then $$B$$ selects $$m$$ images. The first image $$B$$ selects has a chance of $$\frac{n}{t}$$ to overlap with $$A$$'s images. The second image $$B$$ selects has a chance of $$\frac{n}{t-1}$$ provided that the first selected image didn't overlap or a chance of $$\frac{n-1}{t-1}$$ in case it did overlap.

You can calculate all the probabilites based on a tree such as:

The following python code should compute all the probabilities of each node in the tree:

def calc(n=20.0,t=53.0,i=1,f=1.0,d=1):
if i > 20:
return
print " "*d+"overlap: "+str(f*(n/t))
calc(n-1,t-1,i+1,f*(n/t),d+1)
print " "*d+"no-ovlp: "+str(f*(1-(n/t)))
calc(n,t-1,i+1,f*(n/t),d+1)

calc()


This is at least the algorithmic approach. There probably is a more mathematical approach to this I'm unaware of.