Skip to main content
Cross Validated

Return to Answer

added reference
Source Link
knrumsey
knrumsey
  • 8.8k
  • 27
  • 52

This is indeed a problem (one of many) with hypothesis testing. This particular scenario is discussed in detail in the paper by Andrew Gelman and Hal Stern titled The Difference Between "Significant" and "Not Significant" is not Itself Statistically Significant

From the abstract:

The ubiquity of this statistical error leads us to suggest that students and practitioners be made more aware that the difference between “significant” and “not significant” is not itself statistically significant.

From the discussion:

Statistical significance, in some form, is a way to assess the reliability of statistical findings. However, as we have seen, comparisons of the sort, “X is statistically significant but Y is not,” can be misleading.

Reference

Gelman, A., & Stern, H. (2006). The difference between “significant” and “not significant” is not itself statistically significant. The American Statistician, 60(4), 328-331.

This is indeed a problem (one of many) with hypothesis testing. This particular scenario is discussed in detail in the paper by Andrew Gelman and Hal Stern titled The Difference Between "Significant" and "Not Significant" is not Itself Statistically Significant

From the abstract:

The ubiquity of this statistical error leads us to suggest that students and practitioners be made more aware that the difference between “significant” and “not significant” is not itself statistically significant.

From the discussion:

Statistical significance, in some form, is a way to assess the reliability of statistical findings. However, as we have seen, comparisons of the sort, “X is statistically significant but Y is not,” can be misleading.

This is indeed a problem (one of many) with hypothesis testing. This particular scenario is discussed in detail in the paper by Andrew Gelman and Hal Stern titled The Difference Between "Significant" and "Not Significant" is not Itself Statistically Significant

From the abstract:

The ubiquity of this statistical error leads us to suggest that students and practitioners be made more aware that the difference between “significant” and “not significant” is not itself statistically significant.

From the discussion:

Statistical significance, in some form, is a way to assess the reliability of statistical findings. However, as we have seen, comparisons of the sort, “X is statistically significant but Y is not,” can be misleading.

Reference

Gelman, A., & Stern, H. (2006). The difference between “significant” and “not significant” is not itself statistically significant. The American Statistician, 60(4), 328-331.

removed a hyphen
Source Link
knrumsey
knrumsey
  • 8.8k
  • 27
  • 52

This is indeed a problem (one of many) with hypothesis testing. This particular scenario is discussed in detail in the paper by Andrew Gelman and Hal Stern titled The Difference Between "Significant" and "Not Significant" is not Itself Statistically Significant

From the abstract:

The ubiquity of this statistical error leads us to suggest that students and practitioners be made more aware that the difference between “significant” and “not significant” is not itself statisti- callystatistically significant.

From the discussion:

Statistical significance, in some form, is a way to assess the reliability of statistical findings. However, as we have seen, comparisons of the sort, “X is statistically significant but Y is not,” can be misleading.

This is indeed a problem (one of many) with hypothesis testing. This particular scenario is discussed in detail in the paper by Andrew Gelman and Hal Stern titled The Difference Between "Significant" and "Not Significant" is not Itself Statistically Significant

From the abstract:

The ubiquity of this statistical error leads us to suggest that students and practitioners be made more aware that the difference between “significant” and “not significant” is not itself statisti- cally significant.

From the discussion:

Statistical significance, in some form, is a way to assess the reliability of statistical findings. However, as we have seen, comparisons of the sort, “X is statistically significant but Y is not,” can be misleading.

This is indeed a problem (one of many) with hypothesis testing. This particular scenario is discussed in detail in the paper by Andrew Gelman and Hal Stern titled The Difference Between "Significant" and "Not Significant" is not Itself Statistically Significant

From the abstract:

The ubiquity of this statistical error leads us to suggest that students and practitioners be made more aware that the difference between “significant” and “not significant” is not itself statistically significant.

From the discussion:

Statistical significance, in some form, is a way to assess the reliability of statistical findings. However, as we have seen, comparisons of the sort, “X is statistically significant but Y is not,” can be misleading.

Source Link
knrumsey
knrumsey
  • 8.8k
  • 27
  • 52

This is indeed a problem (one of many) with hypothesis testing. This particular scenario is discussed in detail in the paper by Andrew Gelman and Hal Stern titled The Difference Between "Significant" and "Not Significant" is not Itself Statistically Significant

From the abstract:

The ubiquity of this statistical error leads us to suggest that students and practitioners be made more aware that the difference between “significant” and “not significant” is not itself statisti- cally significant.

From the discussion:

Statistical significance, in some form, is a way to assess the reliability of statistical findings. However, as we have seen, comparisons of the sort, “X is statistically significant but Y is not,” can be misleading.