I'm in materials science. I have a fractional factorial experiment. I am investigating the main effect of various treatments on a measured response. The exact details are probably not needed to understand my question so I will simplify things in order to frame my question a more straight forward way (I have 7 factors, but lets just consider 2).

The problem I have is that my replicates are not true replicates, for each set of measurements (with factors set at defined levels) I am only able to make a single batch of material. I then produce individual specimens from that particular batch for testing (say 10 individuals from a batch).

I am aware that the variation within a particular batch is lower than the variation between batches when the treatment factors are kept constant. Nevertheless, economic reasons prevent me from making multiple batches of materials for all cases.

If I ignore this fact and run an ANOVA in R my p-values suggest that the treatment factors have significance than I suspect they really have. I'm using R for the analysis and at the moment I am using something like:

 anova = aov(Response ~ TreatmentA * TreatmentB, data=dataframe)

As a separate experiment I have done some complete replications, e.g. 3 different batches with 10 specimens tested from each batch and not changed the treatment factors. This gives me an indication of batch-to-batch variability.

Is there a way I can plug results of this second experiment as some sort of additional error factor into my first experiment?

As I said, I'm sure the answer to the question is out there but I might be struggling to find the right search term.

  • $\begingroup$ Is there a reason to believe that batch-to-batch variability will be the same in all conditions (at all combinations of treatment factors)? Are you willing to make this assumption? $\endgroup$ – amoeba Oct 13 '16 at 15:39
  • 1
    $\begingroup$ @amoeba Thanks for the response. Good point, it is quite possible that it won't be the same. Some of the treatment factors could potentially increase the variability. I think that the particular set of conditions I chose for the repeats are likely to be among the worst in terms of variability. Is there any other way I could approach this other than making making that assumption? I would prefer to fail to identify a significant factor than to label something as significant that isn't. The experiment is intended to be a screening exercise in order to guide the next steps. $\endgroup$ – Hellfleet Oct 14 '16 at 8:51
  • $\begingroup$ It's an interesting problem. You said you have 7 factors - that's a lot of factors! Even if all of them are binary, this means 2^7 = 128 batches to cover all combinations of levels; if some factors have more than 2 levels then it's even more. Correct? $\endgroup$ – amoeba Oct 14 '16 at 14:46
  • 1
    $\begingroup$ @amoeba I do have 7 factors and they are all binary. However it is a fractional factorial 2^(7-3) resolution IV design. itl.nist.gov/div898/handbook/pri/section3/eqns/2to7m3.txt So I have 16 runs total. All my 2-way interactions are confounded with 2 others. I've tried to separate the most likely interactions. Therefore I'm looking for main effects and evidence of interactions. The idea is a more specific follow up study is then envisioned. $\endgroup$ – Hellfleet Oct 17 '16 at 7:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.