1
$\begingroup$

Suppose I want to compare a book A with a book B to see if a particular word occurs significantly more or less often in either book. If I use the chi-square test (α = 0.05, one degree of freedom, critical value = 3.84), do I always have to formulate a null hypothesis and an alternative hypothesis? Is it not allowed to formulate only the hypotheses from which you expect something?

For example:

Hypothesis: The frequency of the word "blue" in book A differs significantly from the frequency of the word "blue" in book B.

If the significance test is positive (test statistic is higher than the critical value of 3.84), the hypothesis can be confirmed. Otherwise not. Or must a null hypothesis and an alternative hypothesis always be formulated?

If so:

null hypothesis: The frequency of the word "blue" in book A differs not significantly from the frequency of the word "blue" in book B

alternative hypothesis: The frequency of the word "blue" in book A is significantly higher than the frequency of the word "blue" in book B

Is the formulation of the hypotheses correct? Can I say in the alternative hypothesis that something significantly higher or lower occurs (instead of saying that there is "only a difference")? If the result is that the word "blue" occurs significantly more frequently in book B and not, as suspected, in book A, how would you formulate this?

We reject the null hypothesis, but our formulated alternative hypothesis does not agree with what we found in the analysis. There is significance, but not in the meaning of the alternative hypothesis mentioned. Do we then reject both the null hypothesis and the alternative hypothesis? How would the hypotheses in this example be assessed?

And the last question: in this example, the chi-square test is an independence test? We have a 2 × 2 contingency table of observed and expected frequencies. And the goal is to find out if the frequency difference of the word "blue" in two texts is significant. Also: Chi-Square Test of Independence, right?

Improve this question
$\endgroup$
4
  • $\begingroup$ @Dave thank you for your response. If I understood you correctly, one should always write a null hypothesis and an alternative hypothesis? You said you need the p-value to verify significance. But you can also do that with the test statistic if the test statistic is higher than the specified critical value? The following point has not yet been answered. If my alternative hypothesis assumes that something is significantly more frequent, but the significance test shows the opposite (something is significantly less frequent), do I reject the null hypothesis and alternative hypothesis? $\endgroup$
    – Sunkey
    Commented Jan 13, 2023 at 10:12
  • $\begingroup$ How do you determine the critical value if you don't know the distribution of the test statistic under the null hypothesis? Also, consider a t-test and how the critical value is different for a two-sided alternative and a one-sided alternative. Yes, you need both a null and alternative. // Your remaining questions sound like possible questions to post separately. $\endgroup$
    – Dave
    Commented Jan 13, 2023 at 12:55
  • $\begingroup$ @Dave I said that the chi-square test is used under the following conditions: α = 0.05, one degree of freedom, critical value = 3.84. Accordingly, if the test statistic (the value I get from the chi-square test) is higher than the critical value of 3.84, then there is significance. So the calculation of the p-value is only an optional thing? Either I check the significance on the basis of the test statistic (greater than 3.84) or on the basis of the p-value (less than 0.05). I don't understand why you don't answer the question with the alternative hypothesis. That makes me sad. $\endgroup$
    – Sunkey
    Commented Jan 14, 2023 at 13:19
  • $\begingroup$ How do you get that 3.84 is the critical value? $\endgroup$
    – Dave
    Commented Jan 14, 2023 at 14:57

1 Answer 1

Reset to default
0
$\begingroup$

This might be a reasonable use of a $\chi^2$ test. The test aims to compare two proportions, and you want to know about the difference between two proportions (instances of one word out of the total number of words).

However, you need hypotheses in order to calculate the p-value. Remember, a p-value has to do with how unlikely your observed results are if the null hypothesis is true. If there is no null hypothesis, then such a definition makes no sense. An alternative hypothesis also matters. For instance, you would wind up with a different p-value for a test that one proportion is greater than the other than a test that the proportions are unequal. (The usual $\chi^2$ test is inherently the latter, but there is no theoretical issue with testing if one book has a higher proportion of “blue” than the other.)

If your goal is to show the proportions to be the same, you might be interested in equivalence testing.

Some concerns I have about $\chi^2$ testing in this situation:

  1. You know the entirety of each text and could argue that you have the entire population. Why test at all?

  2. There is a lack of independence among the words. English sentences have structure, and a sentence starting a certain way could make it particularly unlikely to have “blue” (or any other color) mentioned, so I am not sold on the words being a sequence of independent binary trials of whether or not the word is “blue” (e.g., independent coin flips where heads is “blue” and tails is any other word, even if the coin is weighted other than 50/50).

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.