Can anyone give a detailed (and clear) explanation of what her "severity" means (isn't it just the power function assessed at different discrepancies taken as null hypothesis?) and how it fits in the statistical testing literature in general?
$\begingroup$
$\endgroup$
3
-
1$\begingroup$ I think even I can answer the question myself, after I finish reading her papers (still reading). I don't understand how this can be too broad. $\endgroup$– user227843Commented Dec 1, 2018 at 18:28
-
2$\begingroup$ Thanks for the edit; I think that improves the question. The question could perhaps be made a little more precise/focused in scope but I also don't see that it's too broad now. I will reopen but I encourage you to tighten the question up a bit further if you can. $\endgroup$– Glen_bCommented Dec 1, 2018 at 23:08
-
$\begingroup$ We have an answer from the originator of the concept herself, which is wonderful. I encourage others to answer, too. Although the basic idea of severity is not difficult, it can be described in different ways. Mayo and her coauthors have been the primary presenters of the idea. There would be value in others presenting it in other ways--just as different textbooks on the same topic can be valuable for different readers. (Mayo has written many articles and two books on severity and its implications, and her presentation is not always the same, but I still would value others' approaches.) $\endgroup$– MarsCommented Oct 21, 2019 at 20:19
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
2
Yes the severity of a statistical claim C is always in relation to a test and an outcome. It's a measure of how well a claim's flaws are put to a test and found absent. A hypothesis C severely passes a test with result x to the extent that a result that is more discordant from C than is x would probability have occurred were C false. Say that a null hypothesis is rejected in a one-sided Normal test of the mean with an outcome that just reaches the significance level of .025. The significant result indicates some discrepancy from the null, but there is a worry someone will make mountains out of molehills. Spoze the power against an alternative mu' is high. Then the severity for inferring mu> mu' is LOW. That's because the probability of observing a larger difference than observed is probable assuming mu' is true. So severity goes in the direction opposite of power when the data lead to a rejection of a null My new book explains all this in clear detail: Statistical Inference as Severe Testing: How to Get beyond the Statistics Wars.
-
4$\begingroup$ Can you define severity without loss of generality in a few basic equations? E.g, the p-value $p = \text{Pr}_0\{t(X) > t(x)\}$, where $t$ is the test-statistic etc. The severity, $s = \dots$? $\endgroup$ Commented Apr 15, 2019 at 2:22
-
$\begingroup$ @innisfree afaict, severity is to beta what p is to alpha. Specifically the probability of seeing something less extreme than the observed value of the statistic under the nearest alternative hypothesis. $\endgroup$ Commented Dec 27, 2022 at 16:20