# Characterizing datasets and estimating statistically significant differences?

I have four datasets: D1, D2, D3 and D4. Each dataset contains elements (different sample size in each dataset) that can be described by 100 categories (1 represents that the element belongs to that category and 0 represents otherwise). So for example,

D1
Element     Category 1       Category 2       Category 3     ....
1               1                0                1
2               0                1                1
3               1                1                1
..
..

D2
Element     Category 1       Category 2       Category 3     ....
1               1                0                0
2               0                0                1
3               1                0                1
..
..

D3
Element     Category 1       Category 2       Category 3     ....
1               1                0                1
2               0                1                0
3               0                1                1
..
..


I would like to test for any statistical differences in the frequencies of the categories in each dataset. Can someone advise on how to approach this?

My initial thought is to get the frequencies of each category and get a single row that represents a dataset like this:

Dataset          f(Category 1)       f(Category 2)       f(Category 3)     ....
D1                  20                    30                   10
D2                  10                    10                   40
D3                  20                    40                   15


And then check to see for any statistical significance but am still not clear on what will be useful in understanding. My question in plain English is to understand the "differences" between the datasets. Any suggestions?

• If I understand your question correctly, a chi square test may be for you. en.wikipedia.org/wiki/Pearson%27s_chi-squared_test Apr 6, 2012 at 6:22
• @RomanLuštrik (+1), make this answer, but I suggest to add some for details. E.g. one has to take into account that we are dealing with a 3x100 table here. Apr 7, 2012 at 7:28
• Related question? What do these chi-square results mean?
– chl
Apr 12, 2012 at 19:04