# Multiple pairwise comparison DOE

I'm going to conduct experiment with 7 products. One subject can evaluate no more than 3 products. The ultimate goal is to compare all 7 products to each other (let say, on means) and rank them. So, full factorial design leads to 35 groups of products (with 3 products in each group) that is too much. I need to reduce it to at least 10.

Can't wrap my mind how to do this. All software that able to generate orthogonal designs need at least two factors with two levels but it seems I have one factor (with 5 levels) or five factors (with 1 level)...

May be someone can advice totally different approach to reliable rank 7 products on means?

You need an incomplete-block design, where each subject (block) sees only a subset of the products. One possibility is a balanced incomplete-block (BIB) design, in which each pair of products occurs together the same number of times. There are also partially balanced (PBIB) designs, where some pairs occur together a fixed number of times, and other pairs occur a different fixed number of times. In a BIB, each product comparison has the same variance, while in a PBIB, there are two different variances for the comparisons.

There are published tables of BIBs and PBIBs, e.g. in Cochran and Cox (1957), Experimental Designs (2nd ed.). From that source, I found a BIB design for your purpose in plan 11.7, p. 472, as follows:

Block Treatments
1    1, 2, 4
2    2, 3, 5
3    3, 4, 6
4    4, 5, 7
5    5, 6, 1
6    6, 7, 2
7    7, 1, 3


So you can do this with 7 subjects, or 14 for two replications of each pair, etc. You say "at least 10" and I think you meant "at most"; but I suggest trying to go for 14 if possible. With too small an experiment, you could find the whole thing a waste because no results are clear.

PS-- I don't get how you came up with "one factor with 5 levels", "5 factors with one level", or a "full factorial with 35 groups". There is only one factor, product, and it has 7 levels.

• Great answer, thank you! PS--For 35 groups (or subjects, in your term), such design where every and each product will once pair with every another product, will result in 35 subjects. I called it "full factorial design", perhaps it's wrong term in this case. For "5 factors" it was misspelling, sorry. Apr 15, 2015 at 18:46
• Hmmmm, for 7 products, the number of pairs is (7 choose 2) = 7 * 6 / 2 = 21. One of the reasons the above design works out is that each block has (3 choose 2) = 3 pairs, so with 7 blocks you have 7 * 3 = 21 pairs represented -- I.e., each possible pair represented once. Oh, but I do see that (7 choose 3) is 35, so that does explain where that number comes from. The BIB design consists of only 1 / 5 of those triples. Apr 15, 2015 at 20:19