6
$\begingroup$

Lets say I have a population consisting of items A, B, and C. I dont know the distribution of items in the population. I sample from this population 15 times (as an example, closer to 400 in my actual data) and count how many times I get each item, giving me something like this:

df = pd.DataFrame({'type': ['A', 'B', 'C'], 'count': [7,4,4]})

  type  count
0    A      7
1    B      4
2    C      4

I repeat this to get a second sample:

df2 = pd.DataFrame({'type': ['A', 'B', 'C'], 'count': [4,10,1]})

  type  count
0    A      4
1    B     10
2    C      1

I want to test whether these 2 samples come from the same population, or whether their frequency distributions are the same. There are posts on the theory behind this (e.g. Test for difference between 2 empirical discrete distributions), but I'm interested in implementation in Python

If I wanted to do a chi square test, would it be as simple as the following?

import scipy.stats
scipy.stats.chisquare(df['count'], df2['count'])

The docs for the chisquare function says the second argument should be expected frequencies of each category. In my mind, this means the distribution in the population, which I dont know. Is it OK to use another set of observed frequencies if I wanted to compare 2 samples?

Improve this question
$\endgroup$
3
  • $\begingroup$ Are the A,B,C nominal categories, or ordinal? I ask (even though it sounds like nominal) because of the mention of discrete distributions (which it would make no sense to raise if the categories were nominal), leading to some ambiguity. $\endgroup$
    – Glen_b
    Commented May 11, 2019 at 1:55
  • $\begingroup$ That would be me misunderstanding the term 'discrete' then (I thought it referred to the counts). But yes, the categories are nominal $\endgroup$
    – Simon
    Commented May 11, 2019 at 3:02
  • $\begingroup$ Okay, then it makes little sense to look at that linked post. You're probably going to want some form of chi-squared test, as you seem to have figured out. In particular, given the phrasing "whether their frequency distributions are the same" suggests you're looking for a test of homogeneity of proportion. $\endgroup$
    – Glen_b
    Commented May 11, 2019 at 6:25

1 Answer 1

Reset to default
6
$\begingroup$

If you are doing a chi-square test for similarity of distributions, then you want to test the hypothesis that the variable "type" is independent of "sample".

This means performing a test of independence with two-way contingency tables. The command you chose (scipy.stats.chisquare) "tests the null hypothesis that the categorical data has the given frequencies", so that's not what you need.

You can concatenate the samples, adding sample variable to the dataframe, write code to create a contingency table, and then use this command: https://docs.scipy.org/doc/scipy-0.15.1/reference/generated/scipy.stats.chi2_contingency.html

There is an example of how it can be used on two columns in a dataframe here: https://codereview.stackexchange.com/questions/96761/chi-square-independence-test-for-two-pandas-df-columns

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ perfect, thanks. Is that test still appropriate if the margin totals are different for the samples? For example, if sample 1 had a total count of 500 and sample 2 had a margin total of 800? because in that case every cell for sample 1 would be larger than sample 2 $\endgroup$
    – Simon
    Commented May 11, 2019 at 4:03
  • 2
    $\begingroup$ @Simon, differences in group counts don't matter. The only requirement for this test is that expected counts in each cell of the contingency (crosstab) table are at least 1 and at least 5 for the majority of the cells. So I should have added that given that you only have 15 draws total per sample (right?), Fisher Exact Test is recommended for small expected counts. There does not seem to be a clean implementation of it in Python for tables that are not 2-by-2. Here is an alternative: stackoverflow.com/questions/25368284/…. $\endgroup$
    – AlexK
    Commented May 11, 2019 at 5:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.