Return to Answer

It depends on what kind of test you’re doing.

For a difference between the values from two distributions, a nonzero difference indicates an effect. This would cover the usual testing of mean differences with $H_0: \mu_0-\mu_1=0$.

However, with testing variances, it is more common to consider the ratio of variances. In this case, the null would be $H_0: \sigma^2_0/\sigma^2_1 =1$. Consequently, it is desired for the confidence interval of that ratio not to include 1. Zero doesn’t come into the mix. A ratio of zero means that the top variance is zero.

So anyone seeing a confidence interval for a ratio of variances of $(0.8, 1.9)$ and calling the effect significant has made a mistake.

The theory behind this is that a confidence interval is an inversion of a hypothesis test. The confidence interval partitions the space of possible values into what would and would not be rejected by an $\alpha$-level test.

Casella and Berger get into this when they talk about interval estimation. Their unfortunate terminology calls these partitions the “rejection region” and “acceptance region” ( even though we do not quite “accept” a null hypothesis).

(I have seen on here that some exotic confidence intervals need not obey this rule, and I will allow someone else to address such details.)

All of this assumes a two-sample comparison. In the one-sample case, we still have the same thinking that we want to check a confidence interval for some surmised value, such as $\mu=\mu_0$ where $\mu_0$ need not be zero.

(The two-sample case could use a null of something other than equality, such as $H_0: \mu_0-\mu_1=6$. Then the interesting question is if the confidence interval contains 6.)

It depends on what kind of test you’re doing.

For a difference between the values from two distributions, a nonzero difference indicates an effect. This would cover the usual testing of mean differences with $H_0: \mu_0-\mu_1=0$.

However, with testing variances, it is more common to consider the ratio of variances. In this case, the null would be $H_0: \sigma^2_0/\sigma^2_1 =1$. Consequently, it is desired for the confidence interval of that ratio not to include 1. Zero doesn’t come into the mix. A ratio of zero means that the top variance is zero.

So anyone seeing a confidence interval for a ratio of variances of $(0.8, 1.9)$ and calling the effect significant has made a mistake.

The theory behind this is that a confidence interval is an inversion of a hypothesis test. The confidence interval partitions the space of possible values into what would and would not be rejected by an $\alpha$-level test.

Casella and Berger get into this when they talk about interval estimation. Their unfortunate terminology calls these partitions the “rejection region” and “acceptance region” (even though we do not quite “accept” a null hypothesis).

(I have seen on here that some exotic confidence intervals need not obey this rule, and I will allow someone else to address such details.)

All of this assumes a two-sample comparison. In the one-sample case, we still have the same thinking that we want to check a confidence interval for some surmised value, such as $\mu=\mu_0$ where $\mu_0$ need not be zero.

(The two-sample case could use a null of something other than equality, such as $H_0: \mu_0-\mu_1=6$. Then the interesting question is if the confidence interval contains 6.)

It depends on what kind of test you’re doing.

For a difference between the values from two distributions, a nonzero difference indicates an effect. This would cover the usual testing of mean differences with $H_0: \mu_0-\mu_1=0$.

However, with testing variances, it is more common to consider the ratio of variances. In this case, the null would be $H_0: \sigma^2_0/\sigma^2_1 =1$. Consequently, it is desired for the confidence interval of that ratio not to include 1. Zero doesn’t come into the mix. A ratio of zero means that the top variance is zero.

So anyone seeing a confidence interval for a ratio of variances of $(0.8, 1.9)$ and calling the effect significant has made a mistake.

The theory behind this is that a confidence interval is an inversion of a hypothesis test. The confidence interval partitions the space of possible values into what would and would not be rejected by an $\alpha$-level test.

Casella and Berger get into this when they talk about interval estimation. Their unfortunate terminology calls these partitions the “rejection region” and “acceptance region” ( even though we do not quite “accept” a null hypothesis.)

(I have seen on here that some exotic confidence intervals need not obey this rule, and I will allow someone else to address such details.)

All of this assumes a two-sample comparison. In the one-sample case, we still have the same thinking that we want to check a confidence interval for some surmised value, such as $\mu=\mu_0$ where $\mu_0$ need not be zero.

(The two-sample case could use a null of something other than equality, such as $H_0: \mu_0-\mu_1=6$. Then the interesting question is if the confidence interval contains 6.)

It depends on what kind of test you’re doing.

For a difference between the values from two distributions, a nonzero difference indicates an effect. This would cover the usual testing of mean differences with $H_0: \mu_0-\mu_1=0$.

However, with testing variances, it is more common to consider the ratio of variances. In this case, the null would be $H_0: \sigma^2_0/\sigma^2_1 =1$. Consequently, it is desired for the confidence interval of that ratio not to include 1. Zero doesn’t come into the mix. A ratio of zero means that the top variance is zero.

So anyone seeing a confidence interval for a ratio of variances of $(0.8, 1.9)$ and calling the effect significant has made a mistake.

The theory behind this is that a confidence interval is an inversion of a hypothesis test. The confidence interval partitions the space of possible values into what would and would not be rejected by an $\alpha$-level test.

Casella and Berger get into this when they talk about interval estimation. Their unfortunate terminology calls these partitions the “rejection region” and “acceptance region” ( even though we do not quite “accept” a null hypothesis).

(I have seen on here that some exotic confidence intervals need not obey this rule, and I will allow someone else to address such details.)

All of this assumes a two-sample comparison. In the one-sample case, we still have the same thinking that we want to check a confidence interval for some surmised value, such as $\mu=\mu_0$ where $\mu_0$ need not be zero.

(The two-sample case could use a null of something other than equality, such as $H_0: \mu_0-\mu_1=6$. Then the interesting question is if the confidence interval contains 6.)

It depends on what kind of test you’re doing.

For a difference between the values from two distributions, a nonzero difference indicates an effect. This would cover the usual testing of mean differences with $H_0: \mu_0-\mu_1=0$.

However, with testing variances, it is more common to consider the ratio of variances. In this case, the null would be $H_0: {\sigma^2_0}{\sigma^2_1} =1$. Consequently, it is desired for the confidence interval of that ratio not to include 1. Zero doesn’t come into the mix. A ratio of zero means that the top variance is zero.

So anyone seeing a confidence interval for a ratio of variances of $(0.8, 1.9)$ and calling the effect significant has made a mistake.

The theory behind this is that a confidence interval is an inversion of a hypothesis test. The confidence interval partitions the space of possible values into what would and would not be rejected by an $\alpha$-level test.

Casella and Berger get into this when they talk about interval estimation. Their unfortunate terminology calls these partitions the “rejection region” and “acceptance region” ( even though we do not quite “accept” a null hypothesis.)

(I have seen on here that some exotic confidence intervals need not obey this rule, and I will allow someone else to address such details.)

All of this assumes a two-sample comparison. In the one-sample case, we still have the same thinking that we want to check a confidence interval for some surmised value, such as $\mu=\mu_0$ where $\mu_0$ need not be zero.

(The two-sample case could using a null of something other than equality, such as $H_0: \mu_0-\mu_1=6$. Then the interesting question is if the confidence interval contains 6.)

It depends on what kind of test you’re doing.

For a difference between the values from two distributions, a nonzero difference indicates an effect. This would cover the usual testing of mean differences with $H_0: \mu_0-\mu_1=0$.

However, with testing variances, it is more common to consider the ratio of variances. In this case, the null would be $H_0: \sigma^2_0/\sigma^2_1 =1$. Consequently, it is desired for the confidence interval of that ratio not to include 1. Zero doesn’t come into the mix. A ratio of zero means that the top variance is zero.

So anyone seeing a confidence interval for a ratio of variances of $(0.8, 1.9)$ and calling the effect significant has made a mistake.

The theory behind this is that a confidence interval is an inversion of a hypothesis test. The confidence interval partitions the space of possible values into what would and would not be rejected by an $\alpha$-level test.

Casella and Berger get into this when they talk about interval estimation. Their unfortunate terminology calls these partitions the “rejection region” and “acceptance region” ( even though we do not quite “accept” a null hypothesis.)

(I have seen on here that some exotic confidence intervals need not obey this rule, and I will allow someone else to address such details.)

All of this assumes a two-sample comparison. In the one-sample case, we still have the same thinking that we want to check a confidence interval for some surmised value, such as $\mu=\mu_0$ where $\mu_0$ need not be zero.

(The two-sample case could use a null of something other than equality, such as $H_0: \mu_0-\mu_1=6$. Then the interesting question is if the confidence interval contains 6.)