Hypothesis testing: formulation and rejection Suppose I want to compare a book A with a book B to see if a particular word occurs significantly more or less often in either book. If I use the chi-square test (α = 0.05, one degree of freedom, critical value = 3.84), do I always have to formulate a null hypothesis and an alternative hypothesis? Is it not allowed to formulate only the hypotheses from which you expect something?
For example:
Hypothesis: The frequency of the word "blue" in book A differs significantly from the frequency of the word "blue" in book B.
If the significance test is positive (test statistic is higher than the critical value of 3.84), the hypothesis can be confirmed. Otherwise not. Or must a null hypothesis and an alternative hypothesis always be formulated?
If so:
null hypothesis: The frequency of the word "blue" in book A differs not significantly from the frequency of the word "blue" in book B
alternative hypothesis: The frequency of the word "blue" in book A is significantly higher than the frequency of the word "blue" in book B
Is the formulation of the hypotheses correct? Can I say in the alternative hypothesis that something significantly higher or lower occurs (instead of saying that there is "only a difference")? If the result is that the word "blue" occurs significantly more frequently in book B and not, as suspected, in book A, how would you formulate this?
We reject the null hypothesis, but our formulated alternative hypothesis does not agree with what we found in the analysis. There is significance, but not in the meaning of the alternative hypothesis mentioned. Do we then reject both the null hypothesis and the alternative hypothesis? How would the hypotheses in this example be assessed?
And the last question: in this example, the chi-square test is an independence test? We have a 2 × 2 contingency table of observed and expected frequencies. And the goal is to find out if the frequency difference of the word "blue" in two texts is significant. Also: Chi-Square Test of Independence, right?
 A: This might be a reasonable use of a $\chi^2$ test. The test aims to compare two proportions, and you want to know about the difference between two proportions (instances of one word out of the total number of words).
However, you need hypotheses in order to calculate the p-value. Remember, a p-value has to do with how unlikely your observed results are if the null hypothesis is true. If there is no null hypothesis, then such a definition makes no sense. An alternative hypothesis also matters. For instance, you would wind up with a different p-value for a test that one proportion is greater than the other than a test that the proportions are unequal. (The usual $\chi^2$ test is inherently the latter, but there is no theoretical issue with testing if one book has a higher proportion of “blue” than the other.)
If your goal is to show the proportions to be the same, you might be interested in equivalence testing.
Some concerns I have about $\chi^2$ testing in this situation:

*

*You know the entirety of each text and could argue that you have the entire population. Why test at all?


*There is a lack of independence among the words. English sentences have structure, and a sentence starting a certain way could make it particularly unlikely to have “blue” (or any other color) mentioned, so I am not sold on the words being a sequence of independent binary trials of whether or not the word is “blue” (e.g., independent coin flips where heads is “blue” and tails is any other word, even if the coin is weighted other than 50/50).
