3
$\begingroup$

I have count data on different languages from comparable corpora.

It looks something like this, where the counts give the number of clauses with that particular word order attested in each language corpus (constructed example):

          SOV   SVO   VSO   0SV OVS VOS
English    3    124    0     0   6   0 
Tagalog    2    14    109    0   3   10
Dutch      56   61     4     0   7   0
Hindi     110   1      2     2   6   9 

I am interested in the differences between languages, i.e. whether these four languages display different or similar word order patterns. I can use chi-square on the whole table, or for separate columns, but I run into trouble with some cells being 0, or < 5. Fisher's exact test is recommended in those cases, but I haven't been able to do this in R so far. If I try it on the whole table, I get FEXACT errors, complaining either that LDKEY or LDSTP are too big or too small. It does not allow me to take just one column, as it needs a table structure of at least 2x2. It allows me to test two columns at a time, but this does not seem appropriate.

Three questions:

  1. Are these methods even appropriate for count data, or do I need something else entirely? I realise this is not a traditional contingency table.

  2. Is there any implementation of Fisher or something comparable (preferably in R) that can be used on the whole table?

  3. Is there something comparable to Fisher that I could use for a single column? Like I said, chi-squire does not seem appropriate due to low values.

EDIT: I realise that my constructed word order example above has such strong associations between language & word order patterns it doesn't matter too much if chi-squared is used (RE: Nick Cox's answer below).

But what about data distributed like this:

   constr1 constr2 constr3 constr4
L1    1      5       20      194
L2    0      4       19      191 
L3    1      8       30      180

Or this:

   constr1 constr2 constr3 
L1    61      166      0      
L2    55      66       2      
L3    55      60       2
L4    54      114      4      
L5    53      98       5

I guess I really want to know whether there is any alternative to assess the whole table rather than conducting multiple Fisher exact tests (I don't own SAS so Peter's option is not available to me).

$\endgroup$
2
  • $\begingroup$ "Skewed" is the wrong word here as it refers statistically to shape of frequency distribution, not to a (strong) association between variables. If your question is whether rows and/or columns of each entire table are different in pattern, then that is in no sense answered by splitting it into lots of 2x2 tables, none of which can see the others. As with your first example, there is no reason not to use chi-square tests so long as you proceed with a little caution about low expected frequencies. (Older literature was often over scary on the problems with small frequencies.) $\endgroup$
    – Nick Cox
    Oct 23, 2014 at 17:30
  • $\begingroup$ Sorry about wrong terminology, I've edited that. OK, thanks for explaining further. My initial search for appropriate statistics started with the biostathandbook, where the author says "I recommend that you always use an exact test (exact test of goodness-of-fit, Fisher's exact test) if the total sample size is less than 1000." in the section on small numbers in chi-square and G-tests: biostathandbook.com/small.html $\endgroup$
    – user32840
    Oct 24, 2014 at 8:59

2 Answers 2

3
$\begingroup$

I don't see anything about your problem that is non-standard for counts of categories. The only thing that is even a little unusual is that you have extremely marked differences between languages.

For your data I get Pearson chi-square of $687.8$ with $15$ d.f. for a test of no association between the variables and the P-value is minutely small. For what it's worth, my program (Stata) reports the P-value as about $7 \times 10^{-137}$.

A good program should indeed flag small expected frequencies, which are the issue rather than small observed frequencies: I see a flag that 4 cells have less than 1 as expected frequency. So, there is a bit of a worry about the P-value, but it is really quite secondary. You could change the P-value by more than 100 orders of magnitude either way, but the message would be the same.

To put it directly, a simple test underlines what is evident just by looking at the frequencies, namely that the languages are very different, which you know any way. If you have some sceptic who doubts that, then a chi-square test provides back-up.

Doing this with Fisher's test is on one level more correct statistically, but it will not change the practical or scientific conclusion one iota.

You have quantitative data that are pertinent to a discussion, but you don't need statistical inference to add gloss. The numbers speak eloquently for themselves, and the details are the interesting part.

Naturally, I am responding to your example, and being firm about what it implies in no way rules out different conclusions for other data.

If there is a predictive model that predicts actual (relative) frequencies, then testing that is a much more interesting question, but you would need to tell us the details.

To respond a little more directly to your question: Fisher's exact test often is impractical once the frequencies stop being very small.

$\endgroup$
0
$\begingroup$

You can use fisher exact test for category variables with more than 2 groups. See the second part of this post handbook of biological statistics for details.

$\endgroup$
2
  • $\begingroup$ Do you mean the section under 'Post-hoc tests', where the author explains how to do multiple Fisher tests for a 4*2 table on termite damage? $\endgroup$
    – user32840
    Oct 23, 2014 at 14:53
  • $\begingroup$ Yes, and see How to do the test -> SAS $\endgroup$
    – Peter
    Oct 23, 2014 at 14:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.