3
$\begingroup$

chi2 seems to be used to compare distributions between categorical features.

Is it possible to use SciPy chisquare function to compare two distributions of continuous data ?

For example :

v1 = np.random.normal(0,10, size=10)
v2 = np.random.normal(0,1, size=10) 
chisquare(v1,v2)

Here what's returned :

Power_divergenceResult(statistic=14184.095704289084, pvalue=0.0)

And if we want to get :

chisquare(v2,v1)

Here what's returned :

Power_divergenceResult(statistic=17.21881425478222, pvalue=0.045397616560846649)

Is it really something we can't do (or get knowledge from the result)?

$\endgroup$
  • 1
    $\begingroup$ Spicy chi-square was a promising start to a title. Alas, it's only a typo. $\endgroup$ – Nick Cox Dec 16 '15 at 16:18
2
$\begingroup$

For comparision of two samples use either Kolmogorov-Smirnov or Anderson–Darling tests, but the second one is more powerful, here's a short article comparing them: http://www.jaqm.ro/issues/volume-6,issue-3/pdfs/1_engmann_cousineau.pdf

But I also see that you compare with N(0,1), so maybe you're always interested in testing normality, thus I would recommend Shapiro–Wilk test.

Moreover, if you're not only interested in simple null hypothesis testing, but you would like to know how far from each other this two distributions are, check Kullback–Leibler divergence (remember that KL is not metric) or Bhattacharyya distance.

$\endgroup$
  • $\begingroup$ Thank you very much for your answer. That's exactly what I needed to know. I think it may require a minimum number of values to be able to tell that a distribution is normal or to get a score we could rely on, right ? $\endgroup$ – DavidK Dec 16 '15 at 17:05
  • $\begingroup$ I'm not sure what are you asking me, but I suppose you're interested in Shapiro-Wilk test that offers best power for a given significance level, see: Power comparisons of Shapiro–Wilk, Kolmogorov–Smirnov, Lilliefors and Anderson–Darling tests [ instatmy.org.my/downloads/e-jurnal%202/3.pdf ] . $\endgroup$ – Adam Przedniczek Dec 16 '15 at 17:17
  • $\begingroup$ Thanks for the links. My question was about how many points (minimum) is needed to be able to say that a distribution is normal. (What if my distribution has only 8 points ? Is the value from the test also powerful with not so many points ?) $\endgroup$ – DavidK Dec 16 '15 at 17:22
  • $\begingroup$ @DavidK "to get a score we could rely on". If you perform NHST (null hypotesis statistical test) in p-value approach, you set significance level mostly to alpha=0.05 and run the test and always get a result. After that you can ask how reliable the result is in the sense of Type I and Type II Errors. Of course the reliability is naturally connected with sample size. Maybe in spare time you should see the book "The Essential Guide to Effect Sizes" by Paul D. Ellis (short, ~ 170 pages). $\endgroup$ – Adam Przedniczek Dec 16 '15 at 17:29
  • $\begingroup$ @DavidK I sent you only the book name, because I don't have time right now to code power calc from scratch, but if you need statistical power you should consider using G*power calculator [ gpower.hhu.de ] (I think that's official site) $\endgroup$ – Adam Przedniczek Dec 16 '15 at 17:42
1
$\begingroup$

The chisquare test cannot be directly used for continuously distributed data. The distribution of the test statistic assumes that we have a number of bins with a large number of observations per bin. (There are different rules of thumb for how many observations per bin, at least 5 is a common recommendation.)

The chisquare test can be used to compare two histograms with equal bin limits, so it works for binned continuously distributed data.

The most common alternatives for goodness-of-fit tests for continuously distributed data are the Anderson-Darling and the Kolmogorov-Smirnov test which are based on the cumulative distribution function instead of the density or probability mass function.

$\endgroup$
  • $\begingroup$ It's not observations we care about; it's expected frequencies. $\endgroup$ – Nick Cox Dec 16 '15 at 16:21
  • $\begingroup$ AFAIK, scipy only has the one sample chisquare test for comparing observed with expected frequencies, but it can be used to compare 2-sample binned frequencies itl.nist.gov/div898/software/dataplot/refman1/auxillar/… $\endgroup$ – Josef Dec 16 '15 at 17:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.