# Hypothesis testing across multiple samples with no independent variable and a categorical outcome

I'm working on DNA sequencing data, and have 8 samples from a single cohort, all samples having undergone the same mutagenic treatment (one group, no independent variable). Six mutation classes (Mut1-Mut6) are found in the samples with varying frequencies/counts. As is quite apparent from the data shown below, Mut4 occurs at a higher frequency than the other mutation classes:

          Mut1   Mut2   Mut3    Mut4   Mut5   Mut6
Sample1   88     27     26      508    51     40
Sample2   438    108    102     1828   184    145
Sample3   334    78     92      1454   151    131
Sample4   146    38     41      448    40     24
Sample5   165    39     54      420    43     37
Sample6   79     20     28      244    39     20
Sample7   133    30     40      404    35     29
Sample8   119    37     54      382    44     40


I would like to show that across the entire cohort, Mut4 occurs at a higher frequency than the other mutations classes, but can't figure out an appropriate way to do this.

To me, this seems like a chi-squared test would be the most appropriate, with a null hypothesis of a flat distribution. Using R, and having read the data in as freq, this can quite easily be tested for a single sample using the chisq.test command:

chisq.test(freq[1,], p=rep(1/6, times=6))


However, I would like to test this across the whole cohort, not just the single sample. I've considered summing all the mutation counts across the cohort, but this would lead to issues with certain samples being weighted more than others (f.ex. sample 2 and 3 have very high mutation counts compared to sample 6). This issue could be mitigated by using proportions instead of count, but I'm unsure if this is a proper way to perform this test.

I've also tried running the chisq.test command on the entire dataset as follows:

chisq.test(freq, p=rep(1/6, times=6))


But I'm unsure of how the results are actually calculated, and am thrown off by the fact that the results seem to be completely unaffected by what I enter as the values for p.

Am I on the right track using a chi-squared, and if so, what is the correct way to move forward from here? Or am I completely off track, and if so, what would be the correct way to test if Mut4 occurs more frequently than the other mutation classes?

• Was the higher frequency of Mut4 a pre-defined hypothesis that you were testing, or was the higher frequency only suggested by the experimental results themselves? – EdM Jan 21 '17 at 1:15
• @EdM the higher frequency of Mut4 was not a pre-defined hypothesis – clfougner Jan 21 '17 at 1:19

Your idea to start with a chi-square test, based on squared differences between observed and expected numbers of cases in each cell, is one correct approach. The issues are what form that test should take, and how to proceed with testing the idea that there is something special about Mut4 as suggested by your data.

A standard chi-square test of a 2-way contingency table calculates the number of cases expected in each cell of the table, under the null hypothesis, from the corresponding row and column totals. An example can be found here. This test, however, examines whether there are associations among rows or columns. Its null hypothesis is that the number of cases in each cell only depends on the corresponding row and column totals as fractions of the total number of cases. It does not test whether there are differences among the columns (in this case, the mutation classes) or among the rows (samples).

You can, however, set up your own chi-square test of your null hypothesis of interest, which is that there are no differences of frequency among the mutation types. In this case, the expected number of cases in each cell would be 1/6 of the total number of cases for the corresponding sample. The p argument to chisq.test() allows for such specific hypotheses (divide expected numbers by total cases, or set rescale.p=TRUE), but the R code for chisq.test (obtained by entering chisq.test at the prompt and hitting return) shows that this argument is ignored for a 2-dimensional table, as you found. Get around that problem by stacking your data into a single vector, providing corresponding expected probabilities in your p argument.

This significant chi-square test, however, only documents that there is a difference from the null hypothesis somewhere; it doesn't say which particular differences are significant. This is as with ANOVA, where a significant F-test doesn't say which particular groups differ. In your case, without a pre-defined hypothesis, you have to correct for testing a result that was suggested by the data, a post-hoc test. This page and this page are reasonable places to start, although with your data a formal test might be considered overkill.

Some type of linear model would also be appropriate. Although count data are often analyzed with generalized linear models based on the Poisson family, with your large numbers of cases a simple ANOVA based on the logarithms of the counts, or on observed frequencies, could work quite well. To test the differences among mutation classes in terms of frequencies you would still need to use an appropriate post-hoc test. Correcting for post-hoc multiple comparisons is often an issue in genomic analyses, so it's worth spending some effort to learn about how the various tests differ.

• Thanks for the response, I'll look into the post-hoc tests you linked to! If I'm understanding your response though, it would be appropriate to run the chi-square test on the entire dataset? However, when I do this, and look at the expected values, I get the following expected values for sample1: 124.95733 31.32297 35.79768 463.4462 47.42357 37.05227, for example. If this were testing the null hypothesis that there is no association between the rows, shouldn't all expected values for one row be equal? This is at least the case when I run the chi-square test on a single row. – clfougner Jan 21 '17 at 11:33
• There's a difference between testing for "no associations between rows and columns," which the standard chi-square test on a 2-way table addresses, and testing "no differences among columns" (among Muts) which is what you're primarily interested in. I didn't address that well in my first crack at an answer. Hope to get a better version up later today or tomorrow. The trick for the initial chi-square will be to use the p argument in a way that takes into account the differences in total numbers of cases among samples, under your null hypothesis of no differences among columns. – EdM Jan 21 '17 at 16:26

Chi2 works under Gaussian assumptions. Firstly it will be nice to convert counts to frequencies. Then I would use a numerical or a non-parametric ANOVA to test H0 that all groups (mutations in your case) have the same mean.

• This doesn't seem to address the question of showing that one particular group has a higher mean than all the others. – whuber Jan 21 '17 at 0:35