Podcast #128: We chat with Kent C Dodds about why he loves React and discuss what life was like in the dark days before Git. Listen now.
3 grammar
source | link

If you are trying to test if distribution depends on the bag -or, equivalently, if all bags are random samples from the same population- performing tests on pairs of bags is not going to work, because it can yield contradictory results -as you found- and because probability of type I errors is going to build up due to the multiple comparisons problem -as Mathew Durry's answer explains and as the XKCD comic demonstrates in a different context.

You can avoid this problem by performing a single test using all bags: a chi-square test for homogeneity, which will tell ifyou whether there are significant differences between bags.

Please notice that most online examples of this test use just a pair of samples, but it works equally fine for more samples. Furthermore, the test is the same as the chi-square test for independence (just interpretation is a bit different), so you can find information under both names.

If homogeneity test shows that there are significant differences between bags, you might be interested on knowing between which bags there are significant differences. Then, paired tests can be useful, but to prevent the multiple comparisons problem to happen again, you need to make corrections. I would suggest Bonferroni correction because of its simplicity.

Anyway, if your bags are just random bags taken from a shop shelf, knowing which one is significantly different is uninteresting and the homogeneity test should be enough for your purposes.

If you are trying to test if distribution depends on the bag -or, equivalently, if all bags are random samples from the same population- performing tests on pairs of bags is not going to work, because it can yield contradictory results -as you found- and because probability of type I errors is going to build up due to the multiple comparisons problem -as Mathew Durry's answer explains and as the XKCD comic demonstrates in a different context.

You can avoid this problem by performing a single test using all bags: a chi-square test for homogeneity, which will tell if there are significant differences between bags.

Please notice that most online examples of this test use just a pair of samples, but it works equally fine for more samples. Furthermore, the test is the same as the chi-square test for independence (just interpretation is a bit different), so you can find information under both names.

If homogeneity test shows that there are significant differences between bags, you might be interested on knowing between which bags there are significant differences. Then, paired tests can be useful, but to prevent the multiple comparisons problem to happen again, you need to make corrections. I would suggest Bonferroni correction because of its simplicity.

Anyway, if your bags are just random bags taken from a shop shelf, knowing which one is significantly different is uninteresting and the homogeneity test should be enough for your purposes.

If you are trying to test if distribution depends on the bag -or, equivalently, if all bags are random samples from the same population- performing tests on pairs of bags is not going to work, because it can yield contradictory results -as you found- and because probability of type I errors is going to build up due to the multiple comparisons problem -as Mathew Durry's answer explains and as the XKCD comic demonstrates in a different context.

You can avoid this problem by performing a single test using all bags: a chi-square test for homogeneity, which will tell you whether there are significant differences between bags.

Please notice that most online examples of this test use just a pair of samples, but it works equally fine for more samples. Furthermore, the test is the same as the chi-square test for independence (just interpretation is a bit different), so you can find information under both names.

If homogeneity test shows that there are significant differences between bags, you might be interested on knowing between which bags there are significant differences. Then, paired tests can be useful, but to prevent the multiple comparisons problem to happen again, you need to make corrections. I would suggest Bonferroni correction because of its simplicity.

Anyway, if your bags are just random bags taken from a shop shelf, knowing which one is significantly different is uninteresting and the homogeneity test should be enough for your purposes.

2 using all bags
source | link

If you are trying to test if distribution depends on the bag -or, equivalently, if all bags are random samples from the same population- performing tests on pairs of bags is not going to work, because it can yield contradictory results -as you found- and because probability of type I errors is going to build up due to the multiple comparisons problem -as Mathew Durry's answer explains and as the XKCD comic demonstrates forin a different context.

You can avoid this problem by performing a single test onusing all bags: a chi-square test for homogeneity, which will tell if there are significant differences between bags.

Please notice that most online examples of this test use just a pair of samples, but it works equally fine for more samples. Furthermore, the test is the same as the chi-square test for independence (just interpretation is a bit different), so you can find information under both names.

If homogeneity test shows that there are significant differences between bags, you might be interested on knowing between which bags there are significant differences. Then, paired tests can be useful, but to prevent the multiple comparisons problem to happen again, you need to make corrections. I would suggest Bonferroni correction because of its simplicity.

Anyway, if your bags are just random bags taken from a shop shelf, knowing which one is significantly different is uninteresting and the homogeneity test should be enough for your purposes.

If you are trying to test if distribution depends on the bag -or, equivalently, if all bags are random samples from the same population- performing tests on pairs of bags is not going to work, because it can yield contradictory results -as you found- and because probability of type I errors is going to build up due to the multiple comparisons problem -as Mathew Durry's answer explains and as the XKCD comic demonstrates for a different context.

You can avoid this problem by performing a single test on all bags: a chi-square test for homogeneity, which will tell if there are significant differences between bags.

Please notice that most online examples of this test use just a pair of samples, but it works equally fine for more samples. Furthermore, the test is the same as the chi-square test for independence (just interpretation is a bit different), so you can find information under both names.

If homogeneity test shows that there are significant differences between bags, you might be interested on knowing between which bags there are significant differences. Then, paired tests can be useful, but to prevent the multiple comparisons problem to happen again, you need to make corrections. I would suggest Bonferroni correction because of its simplicity.

Anyway, if your bags are just random bags taken from a shop shelf, knowing which one is significantly different is uninteresting and the homogeneity test should be enough for your purposes.

If you are trying to test if distribution depends on the bag -or, equivalently, if all bags are random samples from the same population- performing tests on pairs of bags is not going to work, because it can yield contradictory results -as you found- and because probability of type I errors is going to build up due to the multiple comparisons problem -as Mathew Durry's answer explains and as the XKCD comic demonstrates in a different context.

You can avoid this problem by performing a single test using all bags: a chi-square test for homogeneity, which will tell if there are significant differences between bags.

Please notice that most online examples of this test use just a pair of samples, but it works equally fine for more samples. Furthermore, the test is the same as the chi-square test for independence (just interpretation is a bit different), so you can find information under both names.

If homogeneity test shows that there are significant differences between bags, you might be interested on knowing between which bags there are significant differences. Then, paired tests can be useful, but to prevent the multiple comparisons problem to happen again, you need to make corrections. I would suggest Bonferroni correction because of its simplicity.

Anyway, if your bags are just random bags taken from a shop shelf, knowing which one is significantly different is uninteresting and the homogeneity test should be enough for your purposes.

1
source | link

If you are trying to test if distribution depends on the bag -or, equivalently, if all bags are random samples from the same population- performing tests on pairs of bags is not going to work, because it can yield contradictory results -as you found- and because probability of type I errors is going to build up due to the multiple comparisons problem -as Mathew Durry's answer explains and as the XKCD comic demonstrates for a different context.

You can avoid this problem by performing a single test on all bags: a chi-square test for homogeneity, which will tell if there are significant differences between bags.

Please notice that most online examples of this test use just a pair of samples, but it works equally fine for more samples. Furthermore, the test is the same as the chi-square test for independence (just interpretation is a bit different), so you can find information under both names.

If homogeneity test shows that there are significant differences between bags, you might be interested on knowing between which bags there are significant differences. Then, paired tests can be useful, but to prevent the multiple comparisons problem to happen again, you need to make corrections. I would suggest Bonferroni correction because of its simplicity.

Anyway, if your bags are just random bags taken from a shop shelf, knowing which one is significantly different is uninteresting and the homogeneity test should be enough for your purposes.