I assume you have two samples of categorical data and want to know if they were taken from the same distribution. Perhaps most commonly, to compare nominal categorical samples, one can use a chi-squared test.
For the fictitious data below, levels of the categorical variables are
denoted by $1$ through $5,$ but those are not ordered labels.
Suppose we have populations with two different probability distributions
$p_1$ and $p_2$ as follows:
p1 = c(.1,.1,.1,.2,.5) # heavy emphasis on last category
p2 = c(.1,.2,.2,.2,.3) # less heavy
Use R to sample data vectors x1
and x2
(of different sizes)
both from population with probabilities $p_1.$
set.seed(2021)
x1 = sample(1:5,1000,rep=T,prob=p1)
t1 = tabulate(x1, nbins=5); t1
[1] 105 99 85 196 515 # tabulation of first sample
x2 = sample(1:5,1500,rep=T,prob=p1)
t2 = tabulate(x2, nbins=5); t2
[1] 157 154 132 289 768 # tabulation of second
Then take a sample 'y` from the population with probabilities $p_2.$
y = sample(1:5,1500,rep=T,prob=p2)
t = tabulate(y, nbins=5); t
[1] 155 287 320 309 429
Use a chi-squared test to compare samples x1
and x2
, using
the appropriate table TABs
of counts:
TBLs = rbind(t1,t2); TBLs
TBLs
[,1] [,2] [,3] [,4] [,5]
t1 105 99 85 196 515
t2 157 154 132 289 768
With P-value above 5% the test finds no difference between these
two samples from the same population.
chisq.test(TBLs)
Pearson's Chi-squared test
data: TBLs
X-squared = 0.18744, df = 4, p-value = 0.9959
By contrast, with P-value near $0,$ a chi-squared test
overwhelmingly rejects the null hypothesis that the two
samples were taken from the same population.
TBLd = rbind(t2,t); TBLd
TBLd
[,1] [,2] [,3] [,4] [,5]
t2 157 154 132 289 768
t 155 287 320 309 429
chisq.test(TBLd)
Pearson's Chi-squared test
data: TBLd
X-squared = 215, df = 4, p-value < 2.2e-16
The chi-squared statistic $H = \sum_{i=1}^2\sum_{j=1}^5 \frac{(X_{ij} = E_{ij})^2}{E_{ij}}$ (labeled X-squared
in output) can be used as a measure of disagreement between the two samples tested. Here, $i=1,2$ sample rows and $j=1,2,3,4,5$ column categories of the table and the $E_{ij}$ are found in the usual way,
using row and column totals.
For example, in the second chi-squared test above, the expected counts (assuming the null hypothesis to be true) of the ten cells are shown below:
chisq.test(TBLd)$exp
[,1] [,2] [,3] [,4] [,5]
t2 156 220.5 226 299 598.5
t 156 220.5 226 299 598.5
In particular, $E_{11} = \frac{(1500)(302)}{3000} = 156.0.$ (In this simple example, this amounts to the average of $157$ and $155.)$ If all of the $E_{ij} > 5,$ then $H \sim\mathsf{Chisq}(\nu),$ where $n = (r-1)(c-1),$ for a table with $r$ rows and $c$ columns.