What's the distinction between a "Null hypothesis statistical test" and any other test?

Question

A recent hot topic of discussion concerns a journal banning the use of "null hypothesis statistical testing procedures (NHSTPs)" from articles submitted to the journal. I see this term used by some writers, but I don't understand what distinction they are trying to make. Is an NHSTP something different from a "test of hypothesis" or a "significance test"?

Answer 1

Background: The editorial in question is this one from Basic and Applied Social Psychology, a journal with a 2015 impact factor of 1.168, i.e., not highly quotable.

Re: OP question, i.e., Is an NHSTP something different from a "test of hypothesis" or a "significance test"? The applicable editorial statements are

1) "...the null hypothesis significance testing procedure (NHSTP)  is invalid..." [Sic, with alpha = 0.05]
2) "...authors will have to remove all vestiges of the NHSTP (p-values, t-values, F-values, statements about ‘‘significant’’ differences or lack thereof, and so on)."
3) "...confidence intervals [Sic, 95%] also are banned from BASP."
4) "...Bayesian procedures are neither required nor banned from BASP." [Sic, depends on which ones, they are either banned or not.]
5) "Are any inferential statistical procedures required?...No..."

The motivation offered for this is in part "...the $p<.05$ bar is too easy to pass and sometimes serves as an excuse for lower quality research. We hope and anticipate that banning the NHSTP will have the effect of increasing the quality of submitted manuscripts by liberating authors from the stultified structure of NHSTP thinking thereby eliminating an important obstacle to creative thinking."

Answer to OP: These editors would probably claim a test of significance is often an improper test of hypothesis. For example, they state that "...Bayesian proposals that at least somewhat circumvent the Laplacian assumption [Sic, I know nothing a priori]...[such that] there might even be cases where there are strong grounds for assuming that the numbers really are there..." This relates in part to the Fisher versus Neyman and Pearson argument as pointed out above by @Livid and for which the editorial would side with Fisher.

Discussion: I am a firm believer in intellectual humility as a fundamental, and indispensable tenet of scientific method. If I, as a researcher, am not allowed to proceed from an assumption-less initial premise in which all prior theory is disbelieved, then I will lose all of my ability to examine data with a creative and open mind. The premise that all numerical processing must be absolute truth is an exposition of cupidity that is sublime. The only truth is data, and I would humbly paraphrase Box by stating that all models are false, especially and most certainly those that presume that any truth arises from anything that is not identically the data itself. That does not mean that I have to choose between Fisher and Neyman/Pearson, rather that I firmly believe neither premise taken alone, but rather examine things exhaustively until my hypotheses are supported and/or rejected to self-consistency of the ensemble. Only self-consistency can be used as a criterion, as no analysis can reveal an absolute truth.

My way of doing things is not for everyone. Many prefer to plan testing in a rigid controlled experiment design that I would call 'top down'. However, controlled experiments are inefficient for data mining, pattern recognition, and generating hypotheses. They are useful for testing narrow questions, and that is when the controversy about NHSTP can arise. Without supporting evidence, e.g., an entire structure of self-consistency to rely on, any one test is open to criticism. This might be considered as Bonferroni in reverse; if multiple tests lead to an inescapably self-consistent ensemble, the chance of the ensemble occurring by chance alone is diminished. In planning experiments for psychology, the nonsense about not using $p<0.05$ is due to not also testing all of the implications of any particular test outcome, and if one cannot tolerate a type I error of $0.05$ because the experimental design is so rigid, restricted and narrow, then use $0.001$. However, to ban a particular statistical method because it is being used mindlessly and that mindless work passes muster when reviewed merely means that the editors are not identifying low quality work prior to agreeing to review it, and do not approach qualified reviewers. Certainly one cannot establish a reasonable conviction based on a single piece of circumstantial evidence. Rather, an ensemble of circumstantial evidence leads to a reasonable conviction. Eliminating an entire category of evidence because it is circumstantial will not improve a journal's content.