# What can be used to evaluate “variance” of samples of categorical data?

Suppose you are drawing samples from a population of categorical variables. Given samples s1 and s2, I want to calculate an index that would tell me how close I am to the ideal situation where all elements of the sample are equal to each other:

# Given the two samples
pop <- c("A", "B", "C")
set.seed(10)

s1 <- sample(c("A", "B", "C"), 10, rep=T)
s2 <- sample(c("A", "B", "C"), 10, rep=T)

# Suppose also that s3 = c("A", "A", "A")
# and s4 = c("A", "B", "C")


Basically I'd like to find and index f that when applied to a vector with high "variance" such as s4 gives a maximum (or minimum) value and when applied to a vector of characters with low "variance" such as s3 gives a minimum (or maximum) value.

What can I use? I know that in R I can convert factors to numbers using ther number level encoding, then I could calculate the variance, but I'm not sure this is is the method I'd like to use.

• When you say “all elements of the sample are equal to each other”, do you mean how far you are from a uniform distribution, or how far you are from the original/true/supposed distribution? – dblclik Jul 2 '18 at 19:07
• I mean I would like to evaluate how far is the sample from the "ideal" sample. In my case the ideal sample would be a sample where all the categorical variable are equal to one another. Sample s3 in the example is an ideal sample as I have defined it. – mickkk Jul 2 '18 at 19:13
• Could we edit your question into "What can be used to evaluate difference between an observed sample distribution and a theoretic distribution in the case of categorical data?" – Sextus Empiricus Jul 2 '18 at 20:03
• This issue is covered in a number of posts on site, including stats.stackexchange.com/questions/115453/… (and perhaps stats.stackexchange.com/questions/110600/…) – Glen_b Jul 3 '18 at 13:01

## 1 Answer

With categorical data, the variance depends on the mean, which in this case is a proportion. So it is a bit misleading and imprecise to talk about variance when you can simply summarize the data with an intuitive measure: the proportion. If there are more than two categories, a multinomial probability model can be summarized for either sample separately, or the joint sampling distribution of s1 and s2. Here, the probability refers to the proportion of the sample which is A, B, or C in either s1 or s2 and it adds to 1.

You can test for differences in proportions which is also called association in a contingency table. This is done with the Pearson Chi-square test, the Cochrane Mantel Haenszel test, logistic regression, or other methods for independent data. You can also test for proportions of differences which is also called agreement. This is done with a percentage agreement, Cohen's Kappa, or a weighted Kappa if the values are ordinal.