Practical significance and effect size I just started reading an article in Significance by Neil Sheldon (“What does it All Mean?,” Significance, 16:4, 15–17), which discusses the use and abuse of the term “significance” and the recent statements from the ASA in The American Statistician, and discussions thereof. As soon as the first paragraph, I read:

An old argument, one that can be found in R.A. Fisher’s early writings on the subject, makes the point that statistical significance is not the same as practical significance. In more recent years, the practical significance of a result has been quantified by the use of effect sizes. Very roughly, the effect size provides an indication of the practical significance of a finding.

I certainly agree with the first sentence. But the rest??? Is this widely accepted? Maybe I’m not giving enough import to “very roughly,” but from my own perspective, the only time an effect size measures practical significance is when it is zero. 
My reasoning is that the way one should assess practical significance is by serious evaluation of the observed effect from the perspective of the underlying science. To do that, one needs to see the effect in the same units as the response scale. For example, how big a change in SBP, in mmHg, is attributable to that drug? Effect sizes, however, remove the response scale from consideration, and I’d argue that that is the antithesis of judging the true importance of the effect, and substitutes a relative change in standard-deviation units. So instead of quantifying practical significance, we are quantifying how big the effect is relative to how well we can measure it.
All this said, I do see that there are others who seem to support what Sheldon says. Within this forum, for example, I can refer you to https://stats.stackexchange.com/a/271268/52554 and Practical significance, especially with percents: "standard" measure and threshold, among others. Also, I can see an argument for standardizing an effect in another way, as a percentage change—e.g., we might be able to judge the practical significance of decreasing or increasing SBP by, say, 10 percent. 
As a concrete illustration, consider a case where two experimenters conduct a study comparing two methods of producing rivets to be used in assembling airplane wings. It’s important the the diameters of the rivets be kept to a close tolerance. Suppose that both experimenters observe the same average difference of 0.2 mm between the two methods. But one experimenter’s measurements were taken using a wooden ruler, while the second used a micrometer. Thus the error SDs are vastly different, making the first experimenter’s effect size much smaller than the second’s. I argue that the practical importance of the result is the same in both experiments, and should be judged by whether the 0.2mm discrepancy is important.
In summary, I don’t think we are seriously addressing the distinction between statistical and practical significance by replacing one simplistic, highly abused practice with another one that is equally sloppy. Somewhere along the line, people should be thinking about the actual results achieved—-not looking for another way to avoid thinking carefully.
But, my question: When is it appropriate to measure practical significance using an effect-size measure, and why?
 A: The answer to your question is implicit in your question, but can be made explicit: NEVER. Statistical summaries scrub study results of context and background and therefore no matter how useful they are as a component of (or a form of) reasoned argument, they should never be substituted for the thoughtful consideration of a human.
You are absolutely right in saying that practical significance is a matter for the science rather than the statistics. Yet another context-free rule of thumb for inference is the opposite of what is needed because scientists are already too often encouraged to substitute statistical recipes and rules for thoughtful consideration of the available information.
Statisticians should think hard before recommending yet another yardstick or threshold like P<0.05 or the 'small', 'medium' and 'large' effect size categories of Cohen. I have recently arXived a paper in which I dig deeply into the topic of the role of p-values in the scientific inferences of scientists (in the specific context of basic pharmacology) and I think it has relevance to this question. https://arxiv.org/abs/1910.02042
