7
$\begingroup$

First of all I have read the answers to this question, but I'm not happy with them, I feel that they miss the point that I'm willing to address here.

I'm looking at a chi-squared test for independence of two dichotomous variables. Let's say the categories are $A, B$ for variable 1 and $a, b$ for variable 2.

I interpret the test as telling me how far the proportion of $a$ among $A$ is from the proportion of $a$ among $B$. It could be that $P(a|A)<P(a|B)$ and it could be the opposite, and if I look at the usual $\frac{(O-E)^2}{E}$ test, it's going to reject both directions: the most extreme $5\%$ of tables where $P(a|A)<P(a|B)$ as well as the most extreme $5\%$ of tables where $P(a|A)>P(a|B)$, which together make up the most extreme $5\%$ of all tables.

If you're willing to test only one of these directions, to me it makes perfect sense to use a $10\%$ level of significance and reject the null hypothesis only if the inequality goes the way you predicted. The argument that the $\chi^2$ distribution is asymmetrical (has only one tail that encompasses both extreme situations) looks artificial to me: if you really insist that this matters you could pretty much introduce a new statistic called $\pm\chi^2$ that is the same as $\chi^2$, except you add a minus sign when, say, $P(a|A)<P(a|B)$. Then the curve is symmetrical and does the expected job. What is wrong with this point of view?

$\endgroup$
3
  • $\begingroup$ You may have intended $2.5\%$ rather than $10\%$, which keeps the critical point in a normal test near two standard deviations, Fisher's rule of thumb for further investigation. I have sympathy for this, as it discourages arbitrary switching between two- and one-tailed test to get a "more significant" result $\endgroup$
    – Henry
    Commented Mar 3, 2020 at 8:27
  • 1
    $\begingroup$ In some hypothesis tests, a low $\chi^2$ statistic is saying the observed data is excessively close to the expected values. If you reject for this reason (e.g. Mendel's peas), then you are really rejecting the hypothesis that this was a random sample rather than the model was wrong $\endgroup$
    – Henry
    Commented Mar 3, 2020 at 8:28
  • $\begingroup$ @Henry No I really meant $10%$, as in $5%$ in each direction, but you really keep only one of them by stating beforehand that you're going to reject the null hypothesis only if the results go your way. (I don't know where my original answer to your comment is gone) $\endgroup$ Commented Jul 6, 2022 at 20:47

1 Answer 1

1
$\begingroup$

An old question, but for completeness:

Based on my understanding of your question, you're simply specifying an $\alpha=0.10$ error rate, and promoting that if one specifies this error rate (through a 5% error rate in either tail of a two-tailed test), then one should use a 10% error rate in a one tailed test. That makes sense to me.

I think where your confusion may lie is that the chi-squared (vs, say, the z-test of proportions) is an inherently two-tailed test. That is, it tests if there is a difference, but does not allow the specification to determine if the difference is in one direction or the other.

So, for the chi-squared in your example (with $\alpha=0.10$), you would place the 10% error in the upper tail for the "two-tailed" test (a bit of a misnomer in this case, but it is the test that either proportion is larger than the other without a direction being specified). For the z-test of proportions, you'd place the 5% error in both tails for the analogous two-tailed test. For a single tailed test (hypothesizing that a specific proportion is larger), you'd place the 10% error in the specified tail (which again, has no comparable representation in the chi-squared formulation).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.