4
$\begingroup$

H0 is commonly understood to signify the absence of a treatment effect or difference between two groups.

Doesn't this understanding ignore the fact that sample data (being a sample) can never fully accurately reflect the nature of the phenomena being studied?

Doesn't the existence of things like Type II errors require acceptance that statistics don't always reflect reality?

In short: Can H0 be considered a satement about numbers and data, not necessarily about the real world?

Improve this question
$\endgroup$
6
  • $\begingroup$ It seems this issue has been addressed at stats.stackexchange.com/questions/13797/…. How does your question differ, if at all? $\endgroup$
    – whuber ♦
    Commented Nov 10, 2011 at 19:58
  • $\begingroup$ As I see it, that question is more asking about the logic of NHST in general, while I'm asking about the assumptions made within the accepted framework of NHST. $\endgroup$
    – Dan Lurie
    Commented Nov 10, 2011 at 20:15
  • 1
    $\begingroup$ Perhaps you could clarify that distinction, Dan. The present wording seems to focus the interest in the logic. $\endgroup$
    – whuber ♦
    Commented Nov 10, 2011 at 20:59
  • 2
    $\begingroup$ Sorry, do let me clarify. While the other question seems to focus on NHST doing things apparently backwards, I'm specifically interested in what the statistical tests are actually doing vs. what people assume/talk about them doing. Admittedly, it's a subtle distinction, but one that isn't addressed in the other question. $\endgroup$
    – Dan Lurie
    Commented Nov 10, 2011 at 21:21
  • 1
    $\begingroup$ Updated the question in hopes of clarifying things. $\endgroup$
    – Dan Lurie
    Commented Nov 11, 2011 at 0:09

1 Answer 1

Reset to default
6
$\begingroup$

The overall answer is yes, indeed, hypothesis testing is a highly formalised representation that does not reflect reality. Some schools of statistical thought simply do not incorporate hypothesis testing into their paradigm.

Question 1: no, statistics precisely deal with the fact that the data is the realisation of a random phenomenon: in both the Fisher-Neyman-Pearson and Bayesian perspectives, the uncertainty about the data (random) and the answer (not definite) is taken into account. Either as Type I/Type II errors, or as a probability value. In those approaches, the statement is never that $H_0$ is true/false, but that the data "significantly" agrees/disagrees with $H_0$. Obviously, users of such tests may go a long way in misinterpreting the outcome of the test, but I see no point in debating this.

Question 2: a double negative does not help but, no, the concept of Type II errors is precisely including the fact that $H_0$ could be false, in order to evaluate the impact of a wrong decision under both circumstances. This has nothing to do with a "reflection of reality", this is coherent within the statistical model (and the Neyman-Pearson paradigm) but it states nothing about misrepresentations of reality/misspecified models.

Question 3: no and yes. A statistical hypothesis $H_0$ is a question about a probabilistic model observed through an observation from this model (or an alternative one). It is not a question about data in the sense it does require one or two models. However, it is not a question about the real world in that it operates only within the framework of this single or those two model(s).

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.