0
$\begingroup$

I have come across the following problem which confused me quite a lot and would be helpful to get your comments:

Problem

Suppose we are testing a newly programmed random number generator. We use it to produce 1000000 random integers, each either 0 or 1. We get 1s for 500700 times and 0s for 499300 times. We'd like to test the null hypothesis $H_0$: this data comes from a binomial distribution $b(1000000, \frac{1}{2})$. against the alternative hypothesis $H_1$: nope.

First of all, what's the significance level $p$ of this test?

Then, discuss the accuracy of the following descriptions of our test:

  • "We accept $H_0$ at the 0.05 significance level."
  • "The probability that $H_0$ is true is $p$."
  • "If $H_0$ id true, then the number of heads is 700 or more away from 500000 with probability $p$"
  • "If $H_0$ is false, then the number of heads is 700 or more away from 500000 with probability $(1-p)$."
  • "If $H_0$ is true, then the number of heads is less that 700 away from 500000 with probability $(1-p).$"

Things that confuse me:

  • When the author asks: "What is the significance level $p$ of the test?" Do we have to calculate $p$ or should be just explain verbally that the significance level $p$ of this hypothesis test is the probability that we reject $H_0$ when $H_0$ is in fact true.

  • Should I answer the first question in the following way?: "We never accept $H_0$ hypothesis at any significance level. We either reject it or not. We reject $H_0$ at 0.05 significance level if we observe the outcome of the experiment of which the probability of happening is less than 0.05 under $H_0$ and if cannot reject the $H_0$ if we do not observe such outcome, i.e. we do not have sufficient information against $H_0$ hypothesis."

  • Should I answer the second question in the following way?: "We never say the $H_0$ is false with any probability. In the same way as for the previous question we either say that $H_0$ is false, or we cannot say that it is false."

  • Should I answer the third question in the following way?: "This statement does not describe the test accurately, because if $H_0$ is true than the probability that the number of heads is 700 or more way from 500000 could be bigger or smaller than $p$. The probability that the number of heads is 700 or more away from 500000 under $h_0$ is actually $p-value$ of the test and it could be bigger or smaller than the significance level. If $p-value$ is bigger than the significance level then we cannot reject $H_0$ and if it is smaller we have sufficient evidence to reject $h_0$."

  • I have no opinion about this, could you provide any?

  • I have no opinion about this, could you provide any?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.