17
$\begingroup$

I have been asked to propose a course in experimental design for advanced graduate students in agronomy and ecology. I have never taken such a course, and was surprised to find that the course might be more aptly named "Beyond one-way ANOVA", and that it covers material that I learned in an advanced graduate course on statistics for agricultural field experiments (e.g. RCBD, Latin Squares, Contrasts, repeated measurements, and covariates). Perhaps I am confused by the name "Experimental Design" rather than "Analysis of Experimental Results".

I have some ideas about what such a course should contain and would appreciate feedback on how this might be integrated into a statistics curriculum that meets the needs of the students while presenting modern alternatives to named lists of designs and their associated tests.

For example, I can't imagine teaching students to use linear and quadratic contrasts with ANOVA that enforces categorization of continuous variables when I could teach them to compare regression models with linear and quadratic functions. In the second case, they would also learn how to deal with factors that are not experimentally defined discrete values. If anything, I might compare the two approaches.

If I were to teach a course in "Experimental Design" I would really like to emphasize fundamental concepts that are independent of the statistical model being applied, and that would translate more broadly to other problems. This would enable students more flexibility to use modern statistical approaches.

Some of the relevant concepts that do not appear to be covered in the existing course include:

  • hierarchical and mixed models (of which I understand ANOVA and relatives as one example)
  • model comparison (e.g. to replace contrasts)
  • using spatial models instead of blocks as 'factors'
  • replication, randomization, and IID
  • differences among hypothesis testing, p-hacking, and pattern recognition.
  • power analysis through simulation (e.g. recovery of parameters from simulated data sets),
  • pre-registration,
  • use of prior knowledge from published studies and scientific principles.

Are there any courses that currently take such an approach? Any texts books with such a focus?

Improve this question
$\endgroup$
6
  • 1
    $\begingroup$ Did you try googling syllabi on the subjects? There's a ton of them $\endgroup$
    – Aksakal
    Commented Oct 28, 2015 at 17:22
  • 2
    $\begingroup$ The experimental design course I took included RCBD, Latin Squares, Contrasts, factorial designs, linear regression, multiple comparison, replication, randomization, IID and some other topics I don't remember off the top of my head. Your list of concepts is nice but realistically I doubt you have time within a course to cover everything. Mixed models was pretty much a course on its own when I took it in grad school. However it depends on the level of depth you go into each topic. $\endgroup$
    – Sheep
    Commented Oct 28, 2015 at 17:23
  • 1
    $\begingroup$ I agree with @Sheep that your list is nice but probably too much. Although I think mixed model (the basis of it) are essentials in today experimental design. $\endgroup$
    – Emilie
    Commented Oct 28, 2015 at 17:25
  • $\begingroup$ @Sheep part of my confusion is why linear regression, multiple comparison, and contrasts are part of an experimental design class as opposed to being taught in a course on statistical analysis. Maybe I am confused about the scope of such a course. $\endgroup$
    – Abe
    Commented Oct 28, 2015 at 19:41
  • 2
    $\begingroup$ Well the goal of designing an experiment is so that you can analyze the data you collect from the experiment, so these two go hand in hand. You should have an analysis plan in mind when designing the experiment. That is what I was taught at least. The linear regression was review for us but it was the underlying model for many designs. $\endgroup$
    – Sheep
    Commented Oct 28, 2015 at 19:46

1 Answer 1

Reset to default
14
$\begingroup$

Here is a list of some books that I like and which would be good material for such a course:

  • David Cox: Planning of Experiments, Wiley classics, 1992. This is non-mathematical, but not easy! A profound discussion of basic concepts behind design.

  • D. R. Cox & Nancy Reid: The Theory of the Design of Experiments, Chapman & Hall, 2000. More mathematical, but still with focus on basic concepts

  • Rosemary A. Bailey: Design of Comparative Experiments, Cambridge UP, 2008. From the foreword: "My philosophy is that you should not choose an experimental design from a list of named designs. Rather, you should think about all aspects of the current experiment, and then decide on how to put them together appropriately ...".

  • George Casella: Statistical Design, Springer, 2008. Another book looking at old topics with fresh eyes!

  • You could do worse than look at George E. P. Box, J Stuart Hunter and William G. Hunter: Statistics for Experimenters: Design, Innovation and Discovery (second edition, Wiley, 2005) for inspiration.

I would avoid older books looking like a catalog of named designs, and go for one of the above based on fundamental principles. One such book I would avoid is the popular (why?) Douglas C. Montgomery: Design and Analysis of Experiments.

 EDIT 2017

Another topic which could be included is optimal experimental design, with concepts such as D-optimal designs or A-optimal designs. There is now a plethora of books, so difficult to advice, some possibilities:
Optimal Experimental Design with R
Optimal Crossover Designs
Optimal Experimental Design for Non-Linear Models: Theory and Applications
Optimal Design of Experiments: A Case Study Approach

There is a lot of development in this area in R, so have a look at https://CRAN.R-project.org/view=ExperimentalDesign

Improve this answer
$\endgroup$
4
  • 1
    $\begingroup$ +1. Out of curiosity, may I ask why you would avoid Montgomery's textbook? $\endgroup$
    – whuber ♦
    Commented Apr 9, 2017 at 16:25
  • 2
    $\begingroup$ I tried once to teach out of it---did'nt work very well. It have some errors, and looks oldfashioned to me, starting from a catalog of named designs. $\endgroup$
    – kjetil b halvorsen ♦
    Commented Apr 9, 2017 at 16:26
  • $\begingroup$ +1 only for scathing but true remark on Montgomery. Our DoE instructor never really liked the approach of the author even though it was recommended in our course. $\endgroup$
    – User1865345
    Commented Aug 3, 2023 at 4:23
  • $\begingroup$ Pertaining to data, Montgomery had indeed some errors part of which he corrected in the subsequent editions (one of which was pointed out to him by my instructor). $\endgroup$
    – User1865345
    Commented Aug 3, 2023 at 4:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.