H0 is commonly understood to signify the absence of a treatment effect or difference between two groups.

Doesn't this understanding ignore the fact that sample data (being a sample) can never fully accurately reflect the nature of the phenomena being studied?

Doesn't the existence of things like Type II errors require acceptance that statistics don't always reflect reality?

In short: Can H0 be considered a satement about numbers and data, not necessarily about the real world?

  • $\begingroup$ It seems this issue has been addressed at stats.stackexchange.com/questions/13797/…. How does your question differ, if at all? $\endgroup$ – whuber Nov 10 '11 at 19:58
  • $\begingroup$ As I see it, that question is more asking about the logic of NHST in general, while I'm asking about the assumptions made within the accepted framework of NHST. $\endgroup$ – Dan Lurie Nov 10 '11 at 20:15
  • $\begingroup$ Perhaps you could clarify that distinction, Dan. The present wording seems to focus the interest in the logic. $\endgroup$ – whuber Nov 10 '11 at 20:59
  • 1
    $\begingroup$ Sorry, do let me clarify. While the other question seems to focus on NHST doing things apparently backwards, I'm specifically interested in what the statistical tests are actually doing vs. what people assume/talk about them doing. Admittedly, it's a subtle distinction, but one that isn't addressed in the other question. $\endgroup$ – Dan Lurie Nov 10 '11 at 21:21
  • 1
    $\begingroup$ Updated the question in hopes of clarifying things. $\endgroup$ – Dan Lurie Nov 11 '11 at 0:09

The overall answer is yes, indeed, hypothesis testing is a highly formalised representation that does not reflect reality. Some schools of statistical thought simply do not incorporate hypothesis testing into their paradigm.

Question 1: no, statistics precisely deal with the fact that the data is the realisation of a random phenomenon: in both the Fisher-Neyman-Pearson and Bayesian perspectives, the uncertainty about the data (random) and the answer (not definite) is taken into account. Either as Type I/Type II errors, or as a probability value. In those approaches, the statement is never that $H_0$ is true/false, but that the data "significantly" agrees/disagrees with $H_0$. Obviously, users of such tests may go a long way in misinterpreting the outcome of the test, but I see no point in debating this.

Question 2: a double negative does not help but, no, the concept of Type II errors is precisely including the fact that $H_0$ could be false, in order to evaluate the impact of a wrong decision under both circumstances. This has nothing to do with a "reflection of reality", this is coherent within the statistical model (and the Neyman-Pearson paradigm) but it states nothing about misrepresentations of reality/misspecified models.

Question 3: no and yes. A statistical hypothesis $H_0$ is a question about a probabilistic model observed through an observation from this model (or an alternative one). It is not a question about data in the sense it does require one or two models. However, it is not a question about the real world in that it operates only within the framework of this single or those two model(s).


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.