I have a naive question about experiment design (I am not a statistician by trade).

Suppose I am doing social psychology, and I am setting up an experiment like Milgram's in the sixties. Do I really have to formulate a quantitative null hypothesis beforehand?

I do not want to re-interpret the results after the fact, hence the soundness of having a clear H0, but at the same time, it might not be immediately clear what H0 should be. After all, this is "exploratory".

In the case of Milgram, apparently what he did was figure out an "expectation" by surveying people about what they thought would happen in the experiment. I can see how you could use that to formulate a very precise H0: the survey gives you an initial distribution, and you can check the real experiment's distribution against the expected one. It becomes quantitative and very precise, significance can be measured.

But is there a concept of an "exploratory" experiment, where it is not known what should happen, and therefore it is difficult to formulate a clear H0? Should we strive to formulate an "arbitrary" H0 in those cases anyway?


The real question is: do you have a hypothesis? In Milgram's study, the real important information is really descriptive (e.g., how many people made it to 450 volts) rather than hypothesis driven. So, in short, the answer is no, you don't always need a statistical hypothesis beforehand. Note that many people may say that an experiment is inherently designed to test a hypothesis; consequently, this design wouldn't be an "experiment" at all.

On the other hand, if you have any type of hypothesis (e.g., majority of participants will make it to 450 volts, participants will go higher in voltage than psychiatrists would expect, men are more likely to do X), then a statistical hypothesis is warranted.

  • $\begingroup$ Milgram could have formulated H0 as: "out of N participants, less than 10% will go to 450 Volts". $\endgroup$
    – Frank
    Sep 9 '13 at 19:10
  • $\begingroup$ But yes, the question is: what is the statistically honest thing to do when you don't really have a hypothesis? $\endgroup$
    – Frank
    Sep 9 '13 at 19:27
  • $\begingroup$ If you do not have a hypothesis, then statistics are a moot point, are they not? If you have a hypothesis, the question is clear. If you do not have a hypothesis at all, then you are merely talking about descriptive statistics (such as Milgram). $\endgroup$
    – Behacad
    Sep 9 '13 at 19:31
  • 1
    $\begingroup$ Got it. Thanks! You could also bootstrap the hypothesis for the next experiment with the first, descriptive experiment, obviously. $\endgroup$
    – Frank
    Sep 9 '13 at 20:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.