7
$\begingroup$

I am measuring the frequency of consonants in a sample lexicon of a language. The results are as follows:

+-----+-----+-----+-----+----+----+-------+
|  p  |  t  |  k  |  m  | n  | r  | Total |
+-----+-----+-----+-----+----+----+-------+
| 278 | 256 | 122 | 189 | 83 | 72 |  1000 |
+-----+-----+-----+-----+----+----+-------+

It is easy to see that the frequencies are rather different, and $\chi^2$ test for goodness of fit confirms this ($\chi^2$ = 208.5345, df = 5, p < 0.001).

The question I am actually really interested is the following:

The most frequent consonant is p, but is this significant?

In other words, can I claim that p is the most frequent consonant? Is p significantly more frequent than the second-most frequent consonant t, or could the difference between the two due to chance?

What test should I use? Is it ok to simply use a $\chi^2$ test for goodness of fit on the frequencies of p and t only?

$\endgroup$
1
  • $\begingroup$ The end of your last sentence doesn't match the first half of your second last sentence. Please fix. $\endgroup$
    – Glen_b
    Jul 10, 2014 at 10:24

1 Answer 1

2
$\begingroup$

Not directly, because the categories you chose to compare are based on their observed values.

It's still possible to test such a thing using a chi-squared test statistic, but the distribution of the test statistic under the null hypothesis may not be (and I expect is not) well approximated by the distribution that applies when the categories being compared is not based on the observed data.

That is, you need to compute a new distribution for the test statistic.

Please also note that if you've already made one comparison (such as the overall chi-squared) and the decision to make this comparison is conditional on that one, the test is also affected by that conditional decision.


Some details:

Here's the situation as I comprehend what's going on.

There's a contingency table of consonant counts.

We decide to test for equality of proportion among two categories. We can construct a chi-square goodness of fit test in the usual manner, by conditioning on their total:

  p    t     Total
 278  256  | 534

(However, this is effectively a one-tailed test, since we know the observed p-count is greater than the observed t-count.)

Then we can, of course, compute a chi=square test statistic:

> chisq.test(c(278,256))

        Chi-squared test for given probabilities

data:  c(278, 256)
X-squared = 0.9064, df = 1, p-value = 0.3411

The p-value may not mean much, but the statistic is still a measure of the size of the difference between the two.

So how do we generate the distribution under the null? It depends on what we assume about the situation and whether this is a post hoc test.

As an example, let's say we are in the situation where we have the 6-category table without any previous test and we're interested in the question "Is the most common one unusually more common than the second-most common one?" against the null that they're just both coming from a distribution where all 6 categories are equally likely

Then we can easily simulate from the distribution under the null. This:

  chisq.test(sort(table(sample(1:6,1000,repl=TRUE)),decr=TRUE)[1:2])$statistic

generates a single observation from that null. We can repeat that many times to get a sense of what the distribution looks like:

enter image description here

Because of the discreteness, it's a bit hard to tell if that's well approximated by the $\chi^2_1$ distribution or not, but a look at the mean and variance suggests not. If we match by mean and variance, however, an appropriately scaled version of the statistic has approximately the right distribution at a different d.f., particularly in the lower tail (of the p-values, i.e. the upper-tail of the chi-square):

enter image description here

If we now look at the original data:

> pchisq(2.4*chisq.test(c(278,256))$statistic,df=0.8,lower.tail=FALSE)
X-squared 
0.1059023 

This approach suggests a p-value of around 0.1 using a chi-=square statistic and a modified chi-square distribution. If we compute the p-value directly from the simulated distribution, we get a p-value of 0.0993.


In other words, can I claim that p is the most frequent consonant?

While I think it's possible to do something like that, I don't think testing the most popular against the second most popular is necessarily the best approach. We could simply consider, for example, the distribution of the proportion of the largest group under the null of equal proportions.

$\endgroup$
4
  • $\begingroup$ Could you elaborate on "need to compute a new distribution for the test statistic"? I want to know whether I can claim that p is the most frequent consonant. $\endgroup$
    – lotomat
    Jul 10, 2014 at 14:30
  • $\begingroup$ Could you elaborate a little on how the $\chi^2$ statistic would be relevant to this particular question? I don't quite see that. $\endgroup$
    – whuber
    Jul 10, 2014 at 15:11
  • $\begingroup$ @whuber I have added some discussion and details. I don't think this is the best approach, but I think some thing can be done. $\endgroup$
    – Glen_b
    Jul 11, 2014 at 4:21
  • $\begingroup$ Thanks for the clarification. This approach appears to lack power because it does not consider the possibility of a third, or fourth, ... etc. most frequent character affecting the distribution of the mode. The challenge, though, is to develop a reasonable model and an effective alternative hypothesis. $\endgroup$
    – whuber
    Jul 11, 2014 at 12:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.