I am planning on performing data collection on subjects to see how well certain wearable products perform. Let's call the products A, B, and C. The goal is to see if any of the products performs more accurately than the others.

A subject can wear two products at a time with no problem. It is possible to put a third product on the subject, but that third product's accuracy might be compromised by doing so.

Say I have 10 subjects. Obviously, I could run 30 trials, with each subject wearing AB, AC, then BC. However, I do not have the time to run that many trials. Is there an existing experimental design that exists for this situation? If you need more details, please let me know.

  • 1
    $\begingroup$ What are the "products"? What does it mean to "perform more accurately"? I think the design you have describesd might be the way to go, look into BIBD (balanced incomplete block designs). $\endgroup$ – kjetil b halvorsen Mar 17 '17 at 19:26
  • $\begingroup$ How many trials can you afford? $\endgroup$ – Michael Hardy Mar 17 '17 at 19:50
  • $\begingroup$ @kjetilbhalvorsen I can't tell you exactly what the products are, but "perform more accurately" means that it has a high percentage of time during which the value reported by the product is within a certain margin of the actual truth value. I will look up BIBD. $\endgroup$ – swagrov Mar 17 '17 at 20:04
  • $\begingroup$ @MichaelHardy The ideal is 10 trials so each subject only has to perform once. I would not want to go above 20 trials. $\endgroup$ – swagrov Mar 17 '17 at 20:04
  • $\begingroup$ @kjetilbhalvorsen Thank you for the direction! This section on wikipedia is perfect for what I need. $\endgroup$ – swagrov Mar 17 '17 at 20:47

If you need to test each device and combinations, you would need to test all $2^3$ combinations. To test this for each person would take 80 trials, not thirty. (The test plan you have would only show interactions and not the main effects for your devices).

The fewest trials could be run with a $2^{3-1}$ fractional factorial design. Each of the devices would have two levels (worn or not) and each device would count as a factor. With this test, and four runs per person, you would be able to, in 40 tests, have every person test every combination of wearing the devices, including the main effects of only wearing a single device. this fractional factorial design dos have a resolution of III, and generally should be avoided, if possible.

If you can afford it, conducting a full factorial design ($2^3$) for the three factors would yield the most information about main effects, interactions (AB, AC, BC, and ABC).

You may also want to look into the "Taguchi" or "Robust" designs of L4($2^3$) or L8($2^3$), which result in the same number of runs per individual but yield different analysis tools and results. Another option to consider with the Taguchi designs is drastically reducing your test runs by having the individuals in an outer array as a noise factor, and not required to perform every test. As I recall, this could easily cut your trials from 40 to 20, or even 10, if the loss in data resolution is worth the savings in running experiments.

The NIST website has a decent tutorial on choosing and implementing designed experiments: http://www.itl.nist.gov/div898/handbook/pri/section3/pri3.htm and http://www.itl.nist.gov/div898/handbook/pri/section5/pri56.htm


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.