6
$\begingroup$

I've been google-ing around but couldn't find an answer to my question. Any help would be appreciated.

The simplest example of my problem is: Imagine I have an bag of marbles of many different colors. Someone comes along and takes a 'possibly' random sampling of X marbles. Someone comes along later and draws another sampling of marbles. Obviously the distributions of different colors will be different. I'm looking for a test which can tell me how likely these differences were due to chance, or due to a non-random sampling.

In my actual problem I'm looking at amino acid distributions in multi-aligned sequences. We've grouped the patients into two groups (or 'unassignable') based on clinical parameters. We're looking for regions where the sequence distributions are different.

Based on the comments here are a few more details:

The data is a set of 'letters' with no inherent ordering, so most tests like the KS-test are out. There are about ~600 items in any of 20 colors, there are anywhere between 0 and ~400 items of each color (so the distribution is skewed). Group-1 has a random sampling of ~200 items and Group-2 is ~40.

My current method is to do a permutation test. To do this I take all letters from the entire set of sequences and shuffle them. Then I take the first ~200 into Group-1 and the next ~40 into Group-2. I calculate the observed distributions in each group and calculate the Euclidean-distance between the distributions. After ~10,000 shufflings I find the likelihood of getting a distance larger then the observed distance.

Obviously this is not my ideal method ... I don't think the Euc-Distance is the best choice, but I couldn't think of a better one. Any ideas on that front would be welcome too.

Improve this question
$\endgroup$
10
  • $\begingroup$ I can do a permutation test, and that's my current implementation, but I was hoping to find something more elegant. $\endgroup$
    – JudoWill
    Commented Aug 21, 2012 at 14:37
  • $\begingroup$ Do you use a parametric model in that is assumed to hold and you want to different distributions of the same family, or are you looking for something nonparametric? $\endgroup$
    – Momo
    Commented Aug 21, 2012 at 16:07
  • $\begingroup$ Also, do you want to find groups or just establish that the two groups that you have are different in terms of their empirical distribution? $\endgroup$
    – Momo
    Commented Aug 21, 2012 at 16:16
  • $\begingroup$ I'm looking to find if they're different. And either parametric or non-parametric are fine with me. Although my sample sizes are small: ~600 items in total, ~200 in one 'sampling' and ~20 in the other, and the number of items of each color range anywhere from ~500 to ~5. $\endgroup$
    – JudoWill
    Commented Aug 21, 2012 at 16:21
  • $\begingroup$ Ok, thanks. What would be a useful parametric distribution here? A bag of marbles with different colors and no replacement sounds like a multivariate hypergeometric distribution might apply. Would you agree? $\endgroup$
    – Momo
    Commented Aug 21, 2012 at 16:40

1 Answer 1

Reset to default
1
$\begingroup$

If I were approaching this I would:

Try to use a random forest of gradient boosted trees to predict the patients (or patient characteristics of interest) based only on the amino acids. These tools handle categorical inputs. This would allow reduction of the region of interest from being 600-dimension (or whatever) to on the order of 5-30 dimensional. In the much smaller dimensionality data-set you would likely find more textbook approaches to be more successful.

Reference: http://www.journalogy.net/Publication/6491785/feature-selection-with-ensembles-artificial-variables-and-redundancyelimination

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.