# Within sample and between sample categorical variation

I don't really know the correct terminology to ask this question well, so bear with me. I have categorical data with counts and I want a measure of how "diverse" or "spread out" the data is. Variance comes to mind, but I don't know if that applies here.

I have 25 populations, for each population I have allele types for 6 different loci. Each Locus can have a different number of possible alleles. The locus with the maximum number of observed alleles is 160, the locus with the minimum number of observed alleles is 13. In each population I have counts of how many samples had a specific allele.

I would like to be able to make comparisons between populations for a specific loci (I could normalize the counts to get a frequency unless there is something wrong with that approach) and within a population between loci (different number of possible categories). If entropy is a good metric for this what would be the best method of smoothing? Would Total sum of squares be an appropriate metric?

Below is an attempt to illustrate a toy example. I'd like to compare variation between A and B within population A as well as A and A between population 1 and 2.

Population 1
A
A*01 : 100
A*02 :   0
B
B*01 :  20
B*02 :  20
B*03 :  50
B*04 :  10

Population 2
A
A*01 :  10
A*02 :   5
B
B*01 :  15
B*02 :   0
B*03 :   0
B*04 :   0


Original Question : Variance (maybe?) of categorical data

• What are typical values for the population sizes?
– EdM
Commented Aug 8, 2019 at 20:52
• @EdM minimum population size of 62, maximum of 120. Number of counts for a specific locus are twice the population size, as each individual has two alleles Commented Aug 8, 2019 at 20:59

You can think about each of your data points as a sample from a categorical distribution. That is, each of the two alleles for an individual at a locus will have one of $$k$$ possible allele types for that locus, with probability $$p_i$$ of having allele type $$i$$. You suspect that your 25 populations will differ in terms of their categorical distributions for any particular locus and want to compare those populations in terms of some measure of "diversity" at the locus, something like the variance of a univariate distribution. You also would like a similar comparison of diversity within each population among the 6 loci.

You have a choice of several measures of such diversity, with that choice depending on what aspect of diversity you are interested in and wish to explain to your audience.

The Shannon entropy, suggested in an answer to your related question, is a classic choice. For a particular combination of locus and population, it is $$-\sum_{i=1}^k p_i \ln p_i$$. You estimate $$p_i$$ from the fraction of total alleles at the locus in the population sample (total alleles = 2 times the number of individuals) having allele type $$i$$.

Shannnon entropy is well respected but has some potential problems in practice. First, as noted in the above-linked answer, the log term means you need to add some small number to each allele type to deal with populations that lack one or more allele types. Second, the plug-in formula that translates the $$p_i$$ values to an estimate of entropy has an intrinsic downward bias compared with the population value, depending on $$k$$ and the total number of alleles in the sample. The bias is particularly large if the number of total alleles in a population sample at a locus isn't much larger than the number of allele types, as seems to be the case in some parts of your data. There is also the possibility that you have completely missed some allele types so that your value of $$k$$ is an underestimate. These issues are noted, with links to further information, on this page.

Another possibility is to take advantage of the already defined extension of variance to a multivariate categorical distribution, its covariance. A categorical distribution is a multinomial distribution with the same set of $$p_i$$ but only a single trial. The covariance of a categorical distribution is thus a $$k$$ by $$k$$ matrix with each diagonal element equal to $$p_i(1-p_i)$$ and each $$i,j$$ off-diagonal element equal to $$-p_ip_j$$.

One way to reduce the covariance matrix of a multivariate distribution to a single number is to compute its trace, the sum of the diagonal elements. For a categorical distribution, that is: $$\sum_{i=1}^k p_i (1-p_i)$$, of a similar form to the entropy but avoiding the problem of unrepresented allele types in a population.

You might also consider coming up with other measures that both make sense and might be easier to explain. For example: the number of allele types at a locus that contain some pre-defined fraction (say, 50% or 80%) of all alleles at that locus in your sample of a population. That would make sense if you were interested more in the distributions of more-frequent rather than less-frequent allele types. Or if you care more about infrequent allele types, you could come up with a measure that emphasized those (e.g., how many allele types contain the 20% of lowest-frequency allele types).

Whichever estimate of diversity you choose might have a bias from the population value, as for Shannon entropy. I recommend bootstrapping to estimate that bias. For testing hypotheses about differences of your diversity measure among loci or among populations, I recommend bootstrap-based confidence intervals based on a method that takes both bias and asymmetry of estimates into account, like the BCa method. I also recommend comparing two types of bootstrap resampling in this case: re-sampling among alleles, and re-sampling among individuals. If the results are not the same then your allele types within and among loci might not be in Hardy-Weinberg equilibrium for individuals or populations, a result that might inform your interpretation of your analysis.

looks like the frequency tables you have are contingency tables. You can conduct pearson's chi square contingency test. Contingency table is a table that displays the frequencies or counts of two categorical variables. The chi-square test of independence evaluates the null hypothesis that there is no association between the two variables in the population. It does this by comparing the observed frequencies in the contingency table with the frequencies that would be expected if the variables were independent. The test calculates a chi-square statistic, which follows a chi-square distribution under the null hypothesis.