I've been google-ing around but couldn't find an answer to my question. Any help would be appreciated.
The simplest example of my problem is: Imagine I have an bag of marbles of many different colors. Someone comes along and takes a 'possibly' random sampling of X marbles. Someone comes along later and draws another sampling of marbles. Obviously the distributions of different colors will be different. I'm looking for a test which can tell me how likely these differences were due to chance, or due to a non-random sampling.
In my actual problem I'm looking at amino acid distributions in multi-aligned sequences. We've grouped the patients into two groups (or 'unassignable') based on clinical parameters. We're looking for regions where the sequence distributions are different.
Based on the comments here are a few more details:
The data is a set of 'letters' with no inherent ordering, so most tests like the KS-test are out. There are about ~600 items in any of 20 colors, there are anywhere between 0 and ~400 items of each color (so the distribution is skewed). Group-1 has a random sampling of ~200 items and Group-2 is ~40.
My current method is to do a permutation test. To do this I take all letters from the entire set of sequences and shuffle them. Then I take the first ~200 into Group-1 and the next ~40 into Group-2. I calculate the observed distributions in each group and calculate the Euclidean-distance between the distributions. After ~10,000 shufflings I find the likelihood of getting a distance larger then the observed distance.
Obviously this is not my ideal method ... I don't think the Euc-Distance is the best choice, but I couldn't think of a better one. Any ideas on that front would be welcome too.