I read [this] and I usually agree with the view that two-sided p-values are double of one-sided "greater" ones ("lesser" ones being 1-"greater"). But someone told me that "probability distribution of the tables with fixed marginal sums used in Fisher's exact test is not symmetric." Is it right?
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
4
In general it's not symmetric, that's correct. I assume the 2x2 case is specifically intended here (beyond that the notion of tails is problematic).
The distribution of the count in (say) the top left cell is discrete and in general, asymmetric. Specifically, it has a hypergeometric distribution. For some particular two-tailed significance level, different fractions will be "in" each tail.
In some cases it will be symmetric, though. For example the distribution of the number of correct guesses in the experiment of the lady tasting tea is symmetric.
(In relation to the exact test, Fisher wasn't one for alternative hypotheses so such considerations as which tail had what wouldn't have been part of what he'd normally concern himself over, except in so far as it might help him arrive at the calculation of the p-value.)
Here's an example that's asymmetric, and where all the values in the critical region for a 5% test lie in one tail. This is for a table with row totals 15 and 60 and column totals 12 and 63, and where we consider the count in the top-left cell:
If we're in Neyman-Pearson mode (and being conservative with our significance level), you would work out your critical region by adding up from the smallest probabilities of the null distribution to the largest until you got as close as possible to $\alpha$ without exceeding it. We see that the left tail here would never be able to come in, because the lowest probability in the left tail is more than our $\alpha$ of $0.05$ (indeed, even if it was a fair bit smaller than 0.05 it wouldn't because it would still take the cumulative sum over 0.05). Clearly then, this would be a one-tailed case.
If we're in Fisher mode, we take our particular table and add in all tables with equal or smaller probability. If our table is in the right tail and its probability is below 0.05 all the smaller ones would be as well.
So that would also be a p-value where all our two-tailed contributions were from the one-tail.
-
$\begingroup$ Thanks! Is there any way that a one-sided value could be equal to a two-sided value ? It seems to me illogical, but as I am not good enough and not sufficiently well-versed in these kind of stats, I prefer to check. Actually, my problem come that some package function yields values that seemed odd to me, so I need to check this before making any conclusions. $\endgroup$ Sep 20, 2017 at 15:38
-
1$\begingroup$ Quite possibly, it depends on the numbers. Why hide the details of the specific situation you want to know about? (Edit: actually, I have an example, I'll put one in my answer) $\endgroup$– Glen_bSep 20, 2017 at 16:05
-
2$\begingroup$ @AndoJurai: There's more than one way of comparing the "extremeness" of tables in opposite tails - Glen_b's described what I think is the most common. Check the package documentation. $\endgroup$ Sep 22, 2017 at 8:24
-
$\begingroup$ Thanks to both of you. I was not particularly hiding the details, just that before citing a package method as potentially broken, I preferred to check, it's more sort of having some courtesy/regards for package makers, actually, plus it was more a case that I encountered and wondered later about, so I had no data anymore readily available for putting as an example in the question. $\endgroup$ Sep 29, 2017 at 12:48