1
$\begingroup$

I am writing up a paper where I report the results of a set of analyses using Bayesian parameter estimating but am really struggling to come up with synonyms for significant.

Take this sentence.

"The analysis presented above allowed us to estimate whether people given decaffeinated coffee and told truthfully that it was decaffeinated still experienced a significant reduction in caffeine withdrawal."

All my training in stats has been using NHST and the language that goes along with it. The problem is that 'significant' is a perfect word to capture effects that are noteworthy without sensationalising them with adjectives like 'pronounced' or 'large'.

So in co-opting the word 'significant' and giving it a specific meaning frequentism has also prevented its use by other statistical traditions as a neutral descriptor of noteworthy effects

Does anyone have some suggestions?

p.s. 'noteworthy' does not seem to work to my ears.

$\endgroup$
3
  • $\begingroup$ (not addressing your question directly) I would suggest more helpful to use descriptors that do describe the magnitude of the effect (e.g. "large" or "substantial", "moderate" or "small"). This is a key advantage of having effect sizes reported on the original metric and reporting intervals (confidence or credible intervals). I wouldn't view this as "sensationalising" but rather "describing"! $\endgroup$ Oct 2, 2019 at 22:45
  • $\begingroup$ Thank you @James Stanley. This is quite tricky because (i) the scale measuring caffeine withdrawal is not a well-known scale, (ii) the only predictor in the model is a categorical predictor so standardising is not really an option. My only option if I were to use your approach would be to use unstandardised effect sizes and somehow quantify what would be considered a large reduction (e.g. people given caffeinated coffee who were told they were getting caffeinated coffee). I will consult some of the literature and see if that is an option. Thank you again. $\endgroup$
    – llewmills
    Oct 3, 2019 at 3:55
  • $\begingroup$ I'd agree it sounds tricky! Unstandardised effect sizes are fine (I never use standardised effect sizes personally) -- I think the more general question here is interpreting what is a "big enough" magnitude effect to be noteworthy, which is often a problem with these kinds of outcomes. $\endgroup$ Oct 3, 2019 at 19:43

1 Answer 1

4
$\begingroup$

I would recommend leaving words like "significance" out of such descriptions. Be specific instead. For your example sentence (evidently intended for a Discussion section), try something like:

"People given decaffeinated coffee and told truthfully that it was decaffeinated experienced a reduction of X (95% credible interval, Y-Z) in caffeine withdrawal versus those who were not so informed."

The magnitude of the reduction will speak for itself (provided that your measure of caffeine withdrawal is standard) and having 0 outside the credible interval will provide assurance that there is likely to be a true difference.

For the Introduction, again emphasize the magnitude with something like:

We thus designed a study to estimate the magnitude of the difference in caffeine withdrawal symptoms when participants were truthfully told that their beverages would be decaffeinated.

I'm not sure what boilerplate language should be used in the Methods to replace the usual "statistical significance was accepted at p < 0.05 in two-sided tests." Maybe something like "We report 95% credible intervals for our analyses" will do.


I disagree with your premise that "'significant' is a perfect word to capture effects that are noteworthy." Sometimes it just means that you had a large enough study to make a (statistically significant) mountain out of a (practically meaningless) mole-hill. Saying something specific is almost always better than using a generic descriptor.

$\endgroup$
3
  • $\begingroup$ Thank you @EdM. Sorry I should have made it clear that I meant this is for the discussion section of a paper, where, traditionally, you don't requote numbers, but instead try to contextualise the results. This is the context in which I asserted that 'significant' was a perfect word to connote noteworthiness, the non-statistical meaning of the word. $\endgroup$
    – llewmills
    Oct 3, 2019 at 3:48
  • $\begingroup$ The problem is that because of its statistical meaning I can no longer use its non-statistical meaning. $\endgroup$
    – llewmills
    Oct 3, 2019 at 3:56
  • $\begingroup$ @llewmills contextualizing the results will be most effective if you put the magnitude of the result into context, even if you do not wish to re-quote the result. For example, "the [duration/magnitude] of caffeine withdrawal symptoms was reduced in half if participants were truthfully told..." A US-based thesaurus provides many potential synonyms for (5 different meanings of) "significant" from which you could choose, but I always prefer to have things described as quantitatively as possible, even in a Discussion. $\endgroup$
    – EdM
    Oct 3, 2019 at 20:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.