Timeline for How does Fisher calculate his $p$-value?
Current License: CC BY-SA 4.0
19 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Aug 1, 2019 at 9:00 | history | tweeted | twitter.com/StackStats/status/1156852224154394624 | ||
| Jul 30, 2019 at 23:12 | vote | accept | nalzok | ||
| Jul 30, 2019 at 13:21 | answer | added | Scortchi♦ | timeline score: 4 | |
| Jul 30, 2019 at 11:30 | comment | added | Scortchi♦ | @Glen_b: Perhaps not necessary: if there's a modern Fisherian approach - one drawing heavily on Fisher's ideas, but rejecting some (e.g. fiducial inference) & extending or formalizing others - it'll only be detailed in secondary sources. | |
| Jul 30, 2019 at 10:25 | comment | added | Glen_b | Yes, certainly it's necessary to provide something like a quote to support my belief (rather than secondary sources making the same claim I did) -- which is why this isn't an answer. | |
| Jul 30, 2019 at 9:47 | comment | added | Scortchi♦ | @Glen_b: Sure, but - & I should've said this - I'm not aware of Fisher's having commented on the issue at all except in this letter. It's Finney who brings up the matter of correspondence to "discrete subdivision of cases of the other tail" as a criticism of the double-the-one-tailed-p-value approach, though he doesn't suggest any particular alternative approach. | |
| Jul 29, 2019 at 22:38 | comment | added | Glen_b | @Scortchi I wouldn't for a moment suggest that it was something he was always and everywhere consistently insisting on - especially across about six decades - but even so there's still a suggestion of it in that quote as what might otherwise have been done (specifically, the "discrete subdivision of cases of the other tail"). He's saying that alternatively to that you can make an argument that you could argue to halve the significance level and look in the observed tail. | |
| Jul 29, 2019 at 16:20 | comment | added | Scortchi♦ | @Glen_b. I'm not sure Fisher did typically use the likelihood to index extremeness. Yates (1984), JRSS A, 147, "Tests of Significance for 2x2 Contingency Tables", p. 444, quotes Fisher's reply to a letter from D.J Finney asking about two-tailed tests for FET (1946): "I believe I can defend the simple solution of doubling the total probability, not because it corresponds to any discrete subdivision of cases of the other tail, but because it corresponds with halving the probability, supposedly chosen in advance, with which the one observed is to be compared. [...] How does this strike you?". | |
| Jul 29, 2019 at 15:16 | review | Suggested edits | |||
| Jul 29, 2019 at 15:35 | |||||
| Jul 22, 2019 at 14:56 | comment | added | user10619 | At math.arizona.edu/~piegorsch/571A/TR194.pdf: "Defined simply, a P-value is a data-based measure that helps indicate departure from a specified null hypothesis, Ho, in the direction of a specified alternative Ha. Formally, it is the probability of recovering a response as extreme as or more extreme than that actually observed, when Ho is true. (Note that ‘more extreme’ is defined in the context of Ha. For example, when testing Ho: = o vs. Ha: > o, ‘more extreme’ corresponds to values of the test statistic supporting > o .)" | |
| Jul 22, 2019 at 4:24 | comment | added | Glen_b | The lady tasting tea won't do. If you do the 4 vs 4 cups version, there's only one probability to deal with (no ''more extreme" to deal with), and if you extend the number of cups (which Fisher does discuss) and don't require the taster to be perfect (but simply to beat chance) then it's a one-tailed test (so 'more extreme' is otherwise obvious). You need to either go to the two tailed version or the $r\times c$ table version (mentioned in my initial comment above), where some method of deciding more extreme is required. It's there that you see that ordering by likelihood under the null occurs | |
| Jul 21, 2019 at 19:41 | comment | added | nalzok | @Glen_b Can you point me to a typical Fisherian test? I think the lady-tasting-tea experiment misses something: its null hypothesis is "the lady gives random guesses about if the milk went in first", but rejecting it only implies "the lady does not give random guesses", rather than "the lady's guesses are always correct". | |
| Jul 21, 2019 at 14:47 | comment | added | user10619 | @nalzok seems you have a clear understanding of Fisher p test. The calculation is apparently based on simulations and observed t statistics. | |
| Jul 21, 2019 at 7:07 | comment | added | ReneBt | 'how do you define extreme without an alternative hypothesis?' is answered in your question 'as extreme as the actual sample value obtained' | |
| Jul 21, 2019 at 7:02 | comment | added | Glen_b | (This is not presently an answer for two reasons; (i) I am debating whether this should be considered a duplicate, and (ii) if this isn't a duplicate and my comment ('it's based on the likelihood') were to be expanded into an answer, I should like to quote Fisher directly - though I don't expect he will mention the word likelihood specifically in this context.) | |
| Jul 21, 2019 at 6:25 | comment | added | Glen_b | ctd ... will tend to lead to rejection, while a Neyman-Pearson test is designed to have power against a specific alternative (or, more generally, against some specific sequence of alternatives). To me that doesn't make them especially competing notions of testing at all, but tools designed for somewhat different situations (i.e. as Alecos mentions here when discussing work by Spanos, complementary), each good at what they're trying to do, and one may quite reasonably choose one or the other depending on the circumstances. | |
| Jul 21, 2019 at 6:17 | comment | added | Glen_b | Actually, this is explicit in the question P-value: Fisherian vs. contemporary frequentist definitions. (This question was even in the "Related" questions list, which you can presently see in the right hand sidebar -- always a good thing to check.) $\,$ Using likelihood leads to what I see as the central difference between Fisher's approach and the Neyman-Pearson approach: Typically, a Fisher test is an "omnibus" test in the sense that every alternative that lowers likelihood ... ctd | |
| Jul 21, 2019 at 2:01 | comment | added | Glen_b | Fisher typically* uses the likelihood to denote what's more extreme -- lower likelihood is more extreme. *(at least where he doesn't have an explicit test statistic which makes the ordering obvious)... e.g. consider the two tailed version of what's usually called the Fisher exact test (and its extension to $r\times c$ tables), where the tables are unambiguously ordered by their likelihood. | |
| Jul 20, 2019 at 23:35 | history | asked | nalzok | CC BY-SA 4.0 |