Hypergeometric, binomial, or something else for enrichment analysis of potentially shared molecular barcodes between groups? I need to analyze three samples of molecular barcodes, two from blood and one from tissue. My question is which of the two blood samples is more similar to the tissue in terms of molecular barcodes.
The number of unique barcodes in sample BLOOD-A is 3,000, in sample BLOOD-B is 20,000, and in TISSUE is 10,000. The differences between numbers of unique barcodes has no biological meaning, it's just related to the abundance of the speciment. "Unique" means that duplicated barcodes within group have already been deleted. However, between groups barcodes can be shared (so unique within but not between groups). Actually, it is the degree of overlap between blood and tissue barcodes that will ultimately determine similarity. 
The overlap between BLOOD-A and TISSUE is 1,000 barcodes. The overlap between BLOOD-B and TISSUE is 2,000 barcodes. Obviously, BLOOD-A is more similar to TISSUE because 33% of its barcodes are also found in tissue. Conversely, for BLOOD-B only 10% of barcodes are shared with TISSUE. However, due to different sample abundances, note that the total number of unique barcodes in TISSUE (10,000) is only half of that in BLOOD-B (20,000), which means that, in principle, I couldn't get more than 50% BLOOD-B barcodes overlapping to TISSUE. Conversely, for BLOOD-A I could get up to 100%, in principle. Also, note that some barcodes in TISSUE may not be found in either BLOOD-A or BLOOD-B, some may be found in both, and some may be found in only one of the two.
I am getting a bit lost here to come up with a reasonable way to test whether BLOOD-A is significantly more similar to TISSUE than BLOOD-B. It looks like an enrichment analysis problem, but I excluded the hypergeo distribution because barcodes may be found in both BLOOD-A and BLOOD-B. Does anybody have any idea?
Thanks in advance for any help!
Roberto
 A: (Taken to answers as this has gotten too long to be a comment)
It looks to me like the underlying issue is defining what you mean precisely by 'more similar'. 
Once you have an appropriate definition that is meaningful for you in the comparison of overlapping sets of different sizes, the rest will probably follow. 
An obvious measure of correlation for overlapping sets is the overlap coefficient - the size of the intersection over the size of the smaller set. If one set is completely contained in another, it takes the value 1.
A second possibility is the Jaccard Index, which is the intersection over the union. If one set is completely contained within another, it is the ratio of the size of the smaller to the size of the larger.
There are, of course, other possibilities; the point is that you need to identify what you mean when you say 'more similar' or 'less similar' ... in order to try to make progress on then creating a test for it.
You can also conceive of these things sort of as a contingency table with (presumably) a missing cell - 
              In TISSUE      Not in TISSUE
 In BLOOD-A    1000              2000
Not BLOOD-A    9000               ??

If you had some value for the final cell, of course many more measures of association and many other forms of analysis would become possible.
