Return to Answer

The p-value should be considered between a CI and a mean, not two CIs. Indeed, the red point falls entirely outside the blue CI, and the blue point falls entirely outside the red CI.

And it is true that under the null hypothesis such an event would happen 5% of the time:

• 2.5% of the time, you get a point above the 95% CI
• 2.5% of the time, you get a point below the 95% CI

If it is only the whiskers that overlap or touch, then the null hypothesis will produce this result a lot less often than 5%. This is because (to use your example) both the blue population mean would need to be high, and at the same time the red population mean would need to be low (exactly how low would depend on the blue value). You can picture it as a 3D multivariate Gaussian plot, with no skew since the two errors are independent of one another:

Along each axis the probability of falling outside the highlighted region (the CI) is 0.05. But the total probabilities of the blue and pink areas, which gives you P of the two CIs barely touching, is less than 0.05 in your case.

A change of variables from the blue/red axes to the green one will let you integrate this volume using a univariate rather than multivariate Gaussian, and the new variance is the pooled variance from @Sextus-Empiricus's answer.

The p-value should be considered between a CI and a parameter value, not two CIs. Indeed, the red point falls entirely outside the blue CI, and the blue point falls entirely outside the red CI.

And it is true that under the null hypothesis such an event would happen 5% of the time:

• 2.5% of the time, you get a point above the 95% CI
• 2.5% of the time, you get a point below the 95% CI

If it is only the whiskers that overlap or touch, then the null hypothesis will produce this result a lot less often than 5%. This is because (to use your example) both the blue sample would need to be low, and at the same time the red sample would need to be high (exactly how high would depend on the blue value). You can picture it as a 3D multivariate Gaussian plot, with no skew since the two errors are independent of one another:

Along each axis the probability of falling outside the highlighted region (the CI) is 0.05. But the total probabilities of the blue and pink areas, which gives you P of the two CIs barely touching, is less than 0.05 in your case.

A change of variables from the blue/red axes to the green one will let you integrate this volume using a univariate rather than multivariate Gaussian, and the new variance is the pooled variance from @Sextus-Empiricus's answer.

The p-value should be considered between a CI and a mean, not two CIs. Indeed, the red point falls entirely outside the blue CI, and the blue point falls entirely outside the red CI.

And it is true that under the null hypothesis such an event would happen 5% of the time:

• 2.5% of the time, you get a point above the 95% CI
• 2.5% of the time, you get a point below the 95% CI

If it is only the whiskers that overlap or touch, then the null hypothesis will produce this result a lot less often than 5%. This is because (to use your example) both the blue population mean would need to be high, and at the same time the red population mean would need to be low (exactly how low would depend on the blue value). You can picture it as a 3D multivariate Gaussian plot, with no skew since the two errors are independent of one another:

Along each axis the probability of falling outside the highlighted region (the CI) is 0.05. But the total probabilities of the blue and pink areas, which gives you P of the two CIs barely touching, is less than 0.05 in your case.

A change of variables from the blue/red axes to the green one will let you integrate this volume using a univariate rather than multivariate Gaussian, and its effect on variance is where the pooled variance in @Sextus-Empiricus's answer comes from.

The p-value should be considered between a CI and a mean, not two CIs. Indeed, the red point falls entirely outside the blue CI, and the blue point falls entirely outside the red CI.

And it is true that under the null hypothesis such an event would happen 5% of the time:

• 2.5% of the time, you get a point above the 95% CI
• 2.5% of the time, you get a point below the 95% CI

If it is only the whiskers that overlap or touch, then the null hypothesis will produce this result a lot less often than 5%. This is because (to use your example) both the blue population mean would need to be high, and at the same time the red population mean would need to be low (exactly how low would depend on the blue value). You can picture it as a 3D multivariate Gaussian plot, with no skew since the two errors are independent of one another:

Along each axis the probability of falling outside the highlighted region (the CI) is 0.05. But the total probabilities of the blue and pink areas, which gives you P of the two CIs barely touching, is less than 0.05 in your case.

A change of variables from the blue/red axes to the green one will let you integrate this volume using a univariate rather than multivariate Gaussian, and the new variance is the pooled variance from @Sextus-Empiricus's answer.

The p-value should be considered between a CI and a mean, not two CIs. Indeed, the red point falls entirely outside the blue CI, and the blue point falls entirely outside the red CI.

And it is true that under the null hypothesis such an event would happen 5% of the time:

• 2.5% of the time, you get a point above the 95% CI
• 2.5% of the time, you get a point below the 95% CI

If it is only the whiskers that overlap or touch, then the null hypothesis will produce this result a lot less often than 5%. This is because (to use your example) both the blue population mean would need to be high, and at the same time the red population mean would need to be low (exactly how low would depend on the blue value). You can picture it as a 3D multivariate Gaussian plot, with no skew since the two errors are independent of one another:

The total probabilities of the blue and red areas gives you P of the CIs barely touching, which is less than 0.05 in your case.

A change of variables from the blue/red axes to the green one will let you integrate this volume using a univariate rather than multivariate Gaussian, and its effect on variance is where the pooled variance in @Sextus-Empiricus's answer comes from.

The p-value should be considered between a CI and a mean, not two CIs. Indeed, the red point falls entirely outside the blue CI, and the blue point falls entirely outside the red CI.

And it is true that under the null hypothesis such an event would happen 5% of the time:

• 2.5% of the time, you get a point above the 95% CI
• 2.5% of the time, you get a point below the 95% CI

If it is only the whiskers that overlap or touch, then the null hypothesis will produce this result a lot less often than 5%. This is because (to use your example) both the blue population mean would need to be high, and at the same time the red population mean would need to be low (exactly how low would depend on the blue value). You can picture it as a 3D multivariate Gaussian plot, with no skew since the two errors are independent of one another:

Along each axis the probability of falling outside the highlighted region (the CI) is 0.05. But the total probabilities of the blue and pink areas, which gives you P of the two CIs barely touching, is less than 0.05 in your case.

A change of variables from the blue/red axes to the green one will let you integrate this volume using a univariate rather than multivariate Gaussian, and its effect on variance is where the pooled variance in @Sextus-Empiricus's answer comes from.