Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

H0 is commonly understood to signify the absence of a treatment effect or difference between two groups.

Doesn't this understanding ignore the fact that sample data (being a sample) can never fully accurately reflect the nature of the phenomena being studied?

Doesn't the existence of things like Type II errors require acceptance that statistics don't always reflect reality?

In short: Can H0 be considered a satement about numbers and data, not necessarily about the real world?

share|improve this question
It seems this issue has been addressed at…. How does your question differ, if at all? – whuber Nov 10 '11 at 19:58
As I see it, that question is more asking about the logic of NHST in general, while I'm asking about the assumptions made within the accepted framework of NHST. – Dan Lurie Nov 10 '11 at 20:15
Perhaps you could clarify that distinction, Dan. The present wording seems to focus the interest in the logic. – whuber Nov 10 '11 at 20:59
Sorry, do let me clarify. While the other question seems to focus on NHST doing things apparently backwards, I'm specifically interested in what the statistical tests are actually doing vs. what people assume/talk about them doing. Admittedly, it's a subtle distinction, but one that isn't addressed in the other question. – Dan Lurie Nov 10 '11 at 21:21
Updated the question in hopes of clarifying things. – Dan Lurie Nov 11 '11 at 0:09
up vote 4 down vote accepted

The overall answer is yes, indeed, hypothesis testing is a highly formalised representation that does not reflect reality. Some schools of statistical thought simply do not incorporate hypothesis testing into their paradigm.

Question 1: no, statistics precisely deal with the fact that the data is the realisation of a random phenomenon: in both the Fisher-Neyman-Pearson and Bayesian perspectives, the uncertainty about the data (random) and the answer (not definite) is taken into account. Either as Type I/Type II errors, or as a probability value. In those approaches, the statement is never that $H_0$ is true/false, but that the data "significantly" agrees/disagrees with $H_0$. Obviously, users of such tests may go a long way in misinterpreting the outcome of the test, but I see no point in debating this.

Question 2: a double negative does not help but, no, the concept of Type II errors is precisely including the fact that $H_0$ could be false, in order to evaluate the impact of a wrong decision under both circumstances. This has nothing to do with a "reflection of reality", this is coherent within the statistical model (and the Neyman-Pearson paradigm) but it states nothing about misrepresentations of reality/misspecified models.

Question 3: no and yes. A statistical hypothesis $H_0$ is a question about a probabilistic model observed through an observation from this model (or an alternative one). It is not a question about data in the sense it does require one or two models. However, it is not a question about the real world in that it operates only within the framework of this single or those two model(s).

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.