I've been trying to figure out the correct way to calculate the p-value for my data. I originally created a simulation that randomly selected numbers that were greater than or less than a certain number in a specific range given by a dataset. Let me give you an example of what my datasets looked like for clarity:

Expected dataset:

exon_number    number_of_exons
4              20
5              16
2              31
4              15
15             20

Observed dataset:

exon_number    number_of_exons
21             30
15             18
16             20

For each line in my datasets, I randomly selected say 100 numbers between 1 and 20 (for an example from the expected dataset) and determined if the randomly selected number was greater than or less than the exon_number. If it were greater than, I would bin it to the greater than bin. I would do this for all the lines in my datasets and created a total greater than or less than bin for my entire dataset. However, since my datasets were of different sizes, there were a greater amount of greater-thans or less-thans complied for the "expected dataset". Is this problematic? Here are my real results:

               Expected    Observed
Less than      698402      11105
Greater than   918898      13573

I understand that the Fisher's exact test is only for small numbers and should not be used, am I correct? I'm trying to test if the observed data seems to cluster more in the beginning or end of a transcript compared to the expected results.

In that case, I've been using the chi-squared method as below:

import numpy
import scipy.stats
scipy.stats.chisquare([11105, 13573], f_exp=[698402, 918898])

However my output gives me a p-value of 0. Am I doing something wrong? Am I running the test incorrectly? Is my data problematic? I'm new to programming and statistical testing. Any help would be greatly appreciated (and explanations)

  • 1
    $\begingroup$ Why are you doing this in the 1st place? What are you actually trying to figure out? $\endgroup$ May 19, 2015 at 15:04
  • $\begingroup$ I'm trying to determine if there is a consensus for the exon's in the observed data to be in the beginning or end of a transcript $\endgroup$ May 19, 2015 at 15:07
  • $\begingroup$ @gung I also edited my post to include what I'm trying to test $\endgroup$ May 19, 2015 at 15:10
  • $\begingroup$ If we tried to translate that into statistical terms, are you wondering if the exons are uniformly distributed? Or are you wondering if the mean exon number is higher or lower than a fixed value? $\endgroup$ May 19, 2015 at 15:11
  • $\begingroup$ @gung I believe I am trying to determine if the mean exon number is higher or lower than a fixed value $\endgroup$ May 19, 2015 at 15:16

1 Answer 1


You are right that you don't want to use Fisher's exact test here. There isn't anything wrong with using it with large numbers, but it ends up being approximated then, so you lose the 'exact test' advantage that people sometimes want. In addition, Fisher's exact test assumes the marginals are fixed in advance, which isn't true here (and is in fact rarely true).

The reason your chi-squared test is not working properly is that the numbers for the expected and observed are not similar. The expected counts sum to 1617300, whereas the observed sum to 24678. What you really want to compare your observed counts to is the expected proportions. Using your data (and R), here is an example:

chisq.test(x=c(11105, 13573), p=c(698402, 918898)/1617300)
#         Chi-squared test for given probabilities
# data:  c(11105, 13573)
# X-squared = 33.185, df = 1, p-value = 8.381e-09

I do not believe this is the right analysis for your question, though. I suspect you need to something like a Mann-Whitney U-test or Wilcoxon signed rank test.

  • $\begingroup$ Are you allowed to use proportions to compare to your observed results with the chi-squared? $\endgroup$ May 19, 2015 at 16:24
  • $\begingroup$ @christylynn002, a one-sample goodness of fit chi-squared test compares observed counts with counts based on the expected proportions, so yes. I still suspect the chi-squared test isn't actually the right test here, though. Your implementation of the chi-squared test is based on this strange random sampling thing you are doing. So I don't think any resulting chi-squared test will be valid. $\endgroup$ May 19, 2015 at 16:28
  • $\begingroup$ I think I figured out what I need to do. Thank you $\endgroup$ May 19, 2015 at 16:56
  • 1
    $\begingroup$ christylynn002 -- I'm glad you figured out what you need, but perhaps you could explain what you did to resolve your issue -- there are several unresolved problems or potential problems with your explanation of the question that a proper solution might clarify. $\endgroup$
    – Glen_b
    May 20, 2015 at 3:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.