# Testing for population similarity on non-parametric data

I'm looking at similarity of samples for non-ordinal/non-parametric data. I've included a dummy dataset below for reference in the question. Neither A, B, or C is ordinal data: it's entirely categorical. Where C is filled in, it shows a Pareto distribution.

The actual dataset is large and a very small subset of the population has C's data filled in (< 10%). Essentially, the columns are not completed in the dataset. However, I'm trying to determine if there the "shape" of A (and, independently, B) is different for where C is filled in and where C is not filled in.

Phrased another way: "Do id's who have a marked C behave the same way in relation to (either/and A, B) as id's who don't have a marked C". And, "do id's who have a marked C behave the same way (as indicated by A,B) as the entire set of id's?"

My readings[1] indicate that what I am looking for is Pearson's Chi-squared test, as I will be testing a non-parameteric dataset against another dataset (in this case, the same dataset in itself); there is no average/stddev meaningful for A,B, or C.

[1] I'm boning up on analytics stats on the fly; this isn't my area of expertise.

id  A   B   C
1   0   1   a
2   0   0   a
3   0   1   a
4   0   0   a
5   0   1   a
6   1   0   a
7   1   1   a
8   1   0   a
9   1   1   a
10  1   0   b
11  0   1   b
12  0   0   b
13  0   1   b
14  0   0   c
15  1   1   c
16  1   0   c
17  1   1   d
18  1   0   e
19  0   1   f
20  0   0   g
21  0  1  null
22  0  1  null
... null C's continue on for a long time.


a) The question of comparing a sub-group with everyone including itself is logically the same as comparing the subgroup with everyone but that subgroup.

The answer to the second of these questions implies the answer to the first. The problem with the first form (compare us with everyone including us) is you lose independence, and in trying to properly account for it, you end up back where you started, with a test that's functionally equivalent to comparing with 'everyone else but this subgroup'. So just focus on comparing those who filled C out with those who don't.

b) C is decidedly not "Pareto". It's discrete categories. What's the probability of taking any specific exact value (say 'a'?) and what's the probability of being between, but not equal to 'a' and 'b'? In your sample, the first has positive probability and the second zero probability - but for an actual Pareto, it's the other way around.

c) I agree, you probably want a chi-square test; if your alternative is two sided and the expected counts aren't small, that's pretty much the same as anything else you could do.

If your alternative is one-sided then a straight one-tailed proportions test would be used instead.

• Re: (b) Apologies. I meant that the distribution of the values of C is pareto-shaped. Commented Jun 25, 2013 at 16:44
• Paul, what I presume you actually intend is that the ordered categories (a, b, etc) have a decreasing probability as you pass into higher categories; since the distance between the start of each category is arbitrary, there's not really a specific 'shape' in the sense that could be compared with a Pareto; if you impose equispacing it might give an impression akin to a discretized Pareto. (If variable C is not ordered categorical, your presentation of it as if it were is confusing) Commented Jun 25, 2013 at 23:50
• I'm referring to the # of each element of C; the histogram shape, if you will. The probability of drawing a given element out of the "bag" of C's is what I was referring to. But, yes. C itself is not ordered. Commented Jun 26, 2013 at 1:09
• Can you show how 'the probability of an element' relates to a Pareto? If C isn't even ordered then one might as easily put the categories in any order one chose. Certainly you can't even draw a histogram -- only a bar chart. If it's unordered, the categories can be put in any order, so the relative sizes couldn't suggest anything but multinomial. Commented Jun 26, 2013 at 2:04