Alternative for Fisher's exact test for count data in table bigger than 2x2

Question

I have count data on different languages from comparable corpora.

It looks something like this, where the counts give the number of clauses with that particular word order attested in each language corpus (constructed example):

          SOV   SVO   VSO   0SV OVS VOS
English    3    124    0     0   6   0 
Tagalog    2    14    109    0   3   10
Dutch      56   61     4     0   7   0
Hindi     110   1      2     2   6   9 

I am interested in the differences between languages, i.e. whether these four languages display different or similar word order patterns. I can use chi-square on the whole table, or for separate columns, but I run into trouble with some cells being 0, or < 5. Fisher's exact test is recommended in those cases, but I haven't been able to do this in R so far. If I try it on the whole table, I get FEXACT errors, complaining either that LDKEY or LDSTP are too big or too small. It does not allow me to take just one column, as it needs a table structure of at least 2x2. It allows me to test two columns at a time, but this does not seem appropriate.

Three questions:

1. Are these methods even appropriate for count data, or do I need something else entirely? I realise this is not a traditional contingency table.

2. Is there any implementation of Fisher or something comparable (preferably in R) that can be used on the whole table?

3. Is there something comparable to Fisher that I could use for a single column? Like I said, chi-squire does not seem appropriate due to low values.

EDIT: I realise that my constructed word order example above has such strong associations between language & word order patterns it doesn't matter too much if chi-squared is used (RE: Nick Cox's answer below).

But what about data distributed like this:

   constr1 constr2 constr3 constr4
L1    1      5       20      194
L2    0      4       19      191 
L3    1      8       30      180

Or this:

   constr1 constr2 constr3 
L1    61      166      0      
L2    55      66       2      
L3    55      60       2
L4    54      114      4      
L5    53      98       5

I guess I really want to know whether there is any alternative to assess the whole table rather than conducting multiple Fisher exact tests (I don't own SAS so Peter's option is not available to me).

Answer 1

I don't see anything about your problem that is non-standard for counts of categories. The only thing that is even a little unusual is that you have extremely marked differences between languages.

For your data I get Pearson chi-square of $687.8$ with $15$ d.f. for a test of no association between the variables and the P-value is minutely small. For what it's worth, my program (Stata) reports the P-value as about $7 \times 10^{-137}$.

A good program should indeed flag small expected frequencies, which are the issue rather than small observed frequencies: I see a flag that 4 cells have less than 1 as expected frequency. So, there is a bit of a worry about the P-value, but it is really quite secondary. You could change the P-value by more than 100 orders of magnitude either way, but the message would be the same.

To put it directly, a simple test underlines what is evident just by looking at the frequencies, namely that the languages are very different, which you know any way. If you have some sceptic who doubts that, then a chi-square test provides back-up.

Doing this with Fisher's test is on one level more correct statistically, but it will not change the practical or scientific conclusion one iota.

You have quantitative data that are pertinent to a discussion, but you don't need statistical inference to add gloss. The numbers speak eloquently for themselves, and the details are the interesting part.

Naturally, I am responding to your example, and being firm about what it implies in no way rules out different conclusions for other data.

If there is a predictive model that predicts actual (relative) frequencies, then testing that is a much more interesting question, but you would need to tell us the details.

To respond a little more directly to your question: Fisher's exact test often is impractical once the frequencies stop being very small.

Answer 2

You can use fisher exact test for category variables with more than 2 groups. See the second part of this post handbook of biological statistics for details.