I am sure this has been done, but I can't find quite the right approach.

EDIT: Trying to explain better.

The rows of colored boxes below are columns of molecular sequence data -- positions in a protein. The circles represent individuals (species, etc) which have that amino acid at that position. An actual data set consists of many of these positions for the same set of taxa. I want to quantify how well each position (set of traits, A1,A2, etc) is correlated with the trait shared by the species (black or white color of dots, e.g., indicating habitat or other external factor). There is an underlying relationship between the species which is going to drive the major pattern of similarity, and I am trying to find where things differ from that pattern, being instead correlated to the trait (black/white).

In the diagrams and question below tried to reduce this question down to what seems to be the underlying statistical question: generate a score for how well correlated the colored squares are with the black and white circles, without regard to the actual color of the square. (The correlation could be the opposite one position over, as in B1 and B2, but that is still strongly linked to the black/white trait.)

Original description below:

I am trying to find a way to score a suite of traits (colored boxes in figure) by how well they coincide with another categorical trait (black and gray circles below). The categorical traits are shown as binary, but would ideally scale to 3 or more.

One approach would be to generate expected frequencies based on the number of traits and the number of categories, and compare the actual frequencies for each.

To get expectations, for example in scenario A1 one state is present in 1/3rd of the cases, its expectation would be the number of each category times that probability.

enter image description here

My questions are (1) whether this is an accepted way to generate the expected probabilities and (2) what the metric should be for assessing them. I want to have a value for each black/white trait state indicating how strongly it is linked to the colored state.

There is no way to get a zero observed in this situation, because one or the other trait will be found in each category. (For example, in B1 and B2, they should both score as high as possible because the traits sort perfectly to the categories, even though they are in opposite order.) Scenario A4 and B4 should score as low as possible and A1 should also have the highest score.

To put it another way, I am not testing direct correlations ("Do females prefer Democrats?") but rather for each potential preference, which ones separate by category ("What topics of conversation show male/female differences?"). The expectation has to take into account that the associations should score high with either association.

Another thing that might/might not be relevant for summing the trait scores is that, they are not independent: If A and B are traits, then there is an expectation that individual samples (all the 0 and 1 circles) will have an underlying pattern of similarity that is not driven by one- or zero-ness. The point of this metric is to sort out which sets of traits show the most influence of category, even if the majority are driven by some other relationship.

Currently I am assuming I would compare the expected total and the observed for each category using a Chi-squared calculation of (O - E)^2/E and then sum them for each trait?

Does this look like a standard test? I would greatly appreciate any links or suggestions.

  • 1
    $\begingroup$ I really can't follow this. Can you state what you are trying to do more plainly & w/o the jargon that you might expect statisticians want to hear? What are the states & traits here? If you had these scores & metrics, what would you want to do with them? What are your actual data & what is your end goal? $\endgroup$ Apr 7, 2014 at 16:13
  • $\begingroup$ Thanks @gung. You are right I was trying to put it in stats terms that I am not that familiar with. I tried a more up-front explanation. $\endgroup$
    – beroe
    Apr 7, 2014 at 22:34

1 Answer 1


If I understand you right, you have two categorical variables ("box colour" and "dot colour") and you want to know to what extent one predicts the other. If that's right, then a simple evaluation measure here is the mutual information between the box colour and the dot colour.

Mutual information is easy to calculate (see first eqn in the linked page), and it has exactly the property that you want: it doesn't care about which category predicts which other category (it doesn't care whether "orange" predicts "black" or "white"), it only cares whether knowing box-colour reduces your uncertainty about dot-colour.

Pearson's chi-squared test (as you mention) would also work fine, and also has that property. Your wording is a bit difficult but I think you've got the idea right for the "expected" counts that you would use for that.

  • $\begingroup$ Been digesting this a bit and figuring how to apply the mutual information formula. It has me on the track of creating contingency tables, which has helped clarify things, I think. I appreciate the help, and will "check" back as I get it implemented. $\endgroup$
    – beroe
    Apr 16, 2014 at 17:39
  • $\begingroup$ This metric didn't actually work for my exact question, but was a good lead to some tests to try. $\endgroup$
    – beroe
    May 9, 2014 at 21:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.