Skip to main content
added 9 characters in body
Source Link
Glen_b
  • 290.4k
  • 37
  • 652
  • 1.1k

Does rejection of null hypothesis (p<α) for one representative sample imply rejection in a different sample?

Definitely not.

Consider a hypothesis test carried out at level $\alpha$. When $H_0$ is true, you reject a fraction of the time that is $\alpha$ (or for composite null, no more than $\alpha$).

So imagine $H_0$ is true and you reject. You then get a new sample at random from the population. The probability that the next sample would result in rejection would still only be $\alpha$.

Now consider a situation where the null is false, but where the power is 50%. Then the probability that the next sample will reject given the current sample rejected is a toss-up.

In particular, for two random samples taken independently from the same population, the probability of rejection for each is normally independent of a rejection in the other (leaving aside some potential issues with finite samplessmall populations, and so on).

Can I think of these 5 times as obtaining p values less than α?

Yes, the test statistic falling into the rejection region corresponds to a p-value $\leq \alpha$

Does rejection of null hypothesis (p<α) for one representative sample imply rejection in a different sample?

Definitely not.

Consider a hypothesis test carried out at level $\alpha$. When $H_0$ is true, you reject a fraction of the time that is $\alpha$ (or for composite null, no more than $\alpha$).

So imagine $H_0$ is true and you reject. You then get a new sample at random from the population. The probability that the next sample would result in rejection would still only be $\alpha$.

Now consider a situation where the null is false, but where the power is 50%. Then the probability that the next sample will reject given the current sample rejected is a toss-up.

In particular, for two random samples taken independently from the same population, the probability of rejection for each is normally independent of a rejection in the other (leaving aside some potential issues with finite samples, and so on).

Can I think of these 5 times as obtaining p values less than α?

Yes, the test statistic falling into the rejection region corresponds to a p-value $\leq \alpha$

Does rejection of null hypothesis (p<α) for one representative sample imply rejection in a different sample?

Definitely not.

Consider a hypothesis test carried out at level $\alpha$. When $H_0$ is true, you reject a fraction of the time that is $\alpha$ (or for composite null, no more than $\alpha$).

So imagine $H_0$ is true and you reject. You then get a new sample at random from the population. The probability that the next sample would result in rejection would still only be $\alpha$.

Now consider a situation where the null is false, but where the power is 50%. Then the probability that the next sample will reject given the current sample rejected is a toss-up.

In particular, for two random samples taken independently from the same population, the probability of rejection for each is normally independent of a rejection in the other (leaving aside some potential issues with small populations, and so on).

Can I think of these 5 times as obtaining p values less than α?

Yes, the test statistic falling into the rejection region corresponds to a p-value $\leq \alpha$

added 9 characters in body
Source Link
Glen_b
  • 290.4k
  • 37
  • 652
  • 1.1k

Does rejection of null hypothesis (p<α) for one representative sample imply rejection in a different sample?

Definitely not.

Consider a hypothesis test carried out at level $\alpha$. When $H_0$ is true, you reject a fraction of the time that is $\alpha$ (or for composite null, no more than $\alpha$).

So imagine $H_0$ is true and you reject. You then get a new sample at random from the population. The probability that the next sample would result in rejection would still only be $1-\alpha$$\alpha$.

Now consider a situation where the null is false, but where the power is 50%. Then the probability that the next sample will reject given the current sample rejected is a toss-up.

In particular, for two random samples taken independently from the same population (with replacement between taking the two samples if the population is finite), the probability of rejection for each is normally independent of a rejection in the other (leaving aside some potential issues with finite samples, and so on).

Can I think of these 5 times as obtaining p values less than α?

Yes, the test statistic falling into the rejection region corresponds to a p-value $\leq \alpha$

Does rejection of null hypothesis (p<α) for one representative sample imply rejection in a different sample?

Definitely not.

Consider a hypothesis test carried out at level $\alpha$. When $H_0$ is true, you reject a fraction of the time that is $\alpha$ (or for composite null, no more than $\alpha$).

So imagine $H_0$ is true and you reject. The probability that the next sample would result in rejection would be $1-\alpha$.

Now consider a situation where the null is false, but where the power is 50%. Then the probability that the next sample will reject given the current sample rejected is a toss-up.

In particular, for two random samples from the same population (with replacement between taking the two samples if the population is finite), the probability of rejection for each is normally independent of a rejection in the other.

Can I think of these 5 times as obtaining p values less than α?

Yes, the test statistic falling into the rejection region corresponds to a p-value $\leq \alpha$

Does rejection of null hypothesis (p<α) for one representative sample imply rejection in a different sample?

Definitely not.

Consider a hypothesis test carried out at level $\alpha$. When $H_0$ is true, you reject a fraction of the time that is $\alpha$ (or for composite null, no more than $\alpha$).

So imagine $H_0$ is true and you reject. You then get a new sample at random from the population. The probability that the next sample would result in rejection would still only be $\alpha$.

Now consider a situation where the null is false, but where the power is 50%. Then the probability that the next sample will reject given the current sample rejected is a toss-up.

In particular, for two random samples taken independently from the same population, the probability of rejection for each is normally independent of a rejection in the other (leaving aside some potential issues with finite samples, and so on).

Can I think of these 5 times as obtaining p values less than α?

Yes, the test statistic falling into the rejection region corresponds to a p-value $\leq \alpha$

Source Link
Glen_b
  • 290.4k
  • 37
  • 652
  • 1.1k

Does rejection of null hypothesis (p<α) for one representative sample imply rejection in a different sample?

Definitely not.

Consider a hypothesis test carried out at level $\alpha$. When $H_0$ is true, you reject a fraction of the time that is $\alpha$ (or for composite null, no more than $\alpha$).

So imagine $H_0$ is true and you reject. The probability that the next sample would result in rejection would be $1-\alpha$.

Now consider a situation where the null is false, but where the power is 50%. Then the probability that the next sample will reject given the current sample rejected is a toss-up.

In particular, for two random samples from the same population (with replacement between taking the two samples if the population is finite), the probability of rejection for each is normally independent of a rejection in the other.

Can I think of these 5 times as obtaining p values less than α?

Yes, the test statistic falling into the rejection region corresponds to a p-value $\leq \alpha$