1
$\begingroup$

I study linguistics, and I'm conducting a research which compares the ratio between two categories of words in 2 different text corpuses.

Let's say I have corpuses A and B. I created 2 categories of words, Let's call them C and D. I want to calculate the ratio between the count of words in category C and the count of words in category D, and see if the ratio in corpus A is significantly larger than the ratio in corpus B.

What statistical test can I use to test this hypothesis?

It's worth mentioning that the count of words in category C should theoretically depend on the count of words in category D and vice versa.

Thanks in advance

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

First, note that if the ratio of C to D is larger in corpus A, then so will be the proportion. So you can do the standard hypothesis test for difference of proportions.

Let: $$p_A = {C_A\over C_A + D_A}$$ $$p_B = {C_B\over C_B + D_B}$$ $$p = {C\over C + D}$$ These are the proportions of words in each corpus as well as the total for both corpuses.

Then set $$Z = {(p_A - p_B)\over p(1 - p)({1\over C_A + D_A} + {1\over C_B + D_B})}$$

If the number of words from categories C and D is large, then your test statistic should be approximately normal. The null hypothesis is that Z = 0 and the two proportions are equal. If you want a p-value of .05, you can reject the null hypothesis when Z > 1.645.

Improve this answer
$\endgroup$
4
  • $\begingroup$ I'm not sure if I can use this test, since i'm not sure if the proportions are normally distributed. $\endgroup$
    – lior_
    Aug 1, 2016 at 13:38
  • 1
    $\begingroup$ By the central limit theorem, the sampling distribution of a proportion will converge to being normally distributed. Assuming these are large corpuses, the test results using the normality approximation will be almost identical to what the exact results would be. That's why this is the default approach to testing the difference between proportions (see onlinecourses.science.psu.edu/stat414/node/268). If you have at least 50 or so words in each category, the normality approximation will be nearly exact. $\endgroup$
    – Wart
    Aug 1, 2016 at 14:17
  • $\begingroup$ I think that should do it. But theoretically speaking, what if I have only 2 words in each category? $\endgroup$
    – lior_
    Aug 1, 2016 at 14:46
  • $\begingroup$ You can use Fisher's exact test. See this question: stats.stackexchange.com/questions/123609/… $\endgroup$
    – Wart
    Aug 1, 2016 at 18:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.