Data. In R statistical software you can use a chi-squared test of homogeneity
of populations to test your first hypothesis. I will use the data in your first
NIST link as an example:
Noncon =c( 36, 46, 42, 63, 38)
Conform =c(264,254,258,237,262)
DTA = rbind(Noncon,Conform)
chisq.test(DTA)
Initial test of homogeneity. Here is a chi-squared test of homogeneity among the six populations from R statistical software:
Pearson's Chi-squared test
data: DTA
X-squared = 12.131, df = 4, p-value = 0.01641
The P-value 0.016 < 0.05 shows that there are significant differences
among the five populations at the 5% level of significance. (Results are consistent with those in the NIST link.)
Looking at residuals. As a first step toward identifying what the difference(s) may be, you can
can compare observed counts $X_i$ and expected counts $E_i$ by considering the Pearson residuals which are $\sqrt{(X_i - E_i)^2/E_i},$ but retaining the sign of the difference $X_i - E_i.$
NC.test = chisq.test(DTA)
NC.test$obs
[,1] [,2] [,3] [,4] [,5]
Noncon 36 46 42 63 38
Conform 264 254 258 237 262
NC.test$exp
[,1] [,2] [,3] [,4] [,5]
Noncon 45 45 45 45 45
Conform 255 255 255 255 255
NC.test$resi
[,1] [,2] [,3] [,4] [,5]
Noncon -1.3416408 0.14907120 -0.4472136 2.683282 -1.043498
Conform 0.5636019 -0.06262243 0.1878673 -1.127204 0.438357
Residuals with absolute values greater than about $2$ may point the way to
interesting differences among populations. Here, we look at Population 4,
where we would have 'expected' $45$ nonconforming specimens (if the null hypothesis were true), but observed $63.$
Looking at the proportions of nonconforming specimens, we have:
Noncon/(Noncon+Conform)
[1] 0.1200000 0.1533333 0.1400000 0.2100000 0.1266667
So Population 4 seems to have a 21% nonconforming specimens, whereas the other four populations all have below 16% nonconforming.
Ad hoc tests comparing pairs of populations. As a first formal test, it makes sense to compare Population 4 with the
Population 2 which has the second greatest proportion of nonconforming specimens: In R, prop.test makes this comparison, and finds no significant difference. (I prefer not to make the 'continuity correction', hence the parameter cor=F.)
prop.test(c(46,63), c(300,300), cor=F)
2-sample test for equality of proportions
without continuity correction
data: c(46, 63) out of c(300, 300)
X-squared = 3.24, df = 1, p-value = 0.07186
alternative hypothesis: two.sided
95 percent confidence interval:
-0.118202692 0.004869359
prop 1 prop 2
0.1533333 0.2100000
The next lower percentage of nonconforming specimens is in Population 3, which is significant, if we test at the 5% level. However, making multiple comparisons at the 5% level may lead to 'false discovery'.
prop.test(c(42,63), c(300,300), cor=F)$p.val
[1] 0.02405158
Next in line is Population 6, which differs from Population 5 at the
1% level. Using the Bonferroni method of avoiding false discovery with
as many as five such comparisons, we can feel confident rejecting at the 1% level.
prop.test(c(38,63), c(300,300), cor=F)$p.val
[1] 0.006376778
In summary, we might say that Population 6 has differs from Populations 1 and 5, possibly from Population 4, and not from Population 2.
It does not seem fruitful to make comparisons among Populations 1, 2, 3, 4, and 6. Mainly, I say this because of Pearson residuals of small absolute value in the first test, but also because these differences may not be of practical importance even if borderline significant.
(however, opinions differ about criteria for such ad hoc comparisons.)
