# Asking nature a single question or a carefully thought out questionnaire

In The Seven Pillars of Statistical Wisdom book by Stephen M. Stigler:

William Stanley Jevons, writing in his Principles of Science in 1874:

One of the most requisite precautions in experimentation is to vary only one circumstance at a time, and to maintain all other circumstances rigidly unchanged.

Ronald A. Fisher, writing in 1926:

No aphorism is more frequently repeated in connection with field trials, than that we must ask Nature few questions, or, ideally, one question, at a time. The writer [Fisher] is convinced that this view is wholly mistaken. Nature, he suggests, will best respond to a logical and carefully thought out questionnaire; indeed, if we ask her a single question, she will often refuse to answer until some other topic has been discussed.

I continued to finish the book and yet can't understand, what was the insight Fisher got. When googling for an answer it seems the former idea (keep everything fixed and change one thing aka ceteris paribus) is the recommended approach.

What is the insight Fisher is talking about?

Think of a simple example, and since RA Fisher's primary experience with experimental design was at Rothamsted Experimental station, let us use an agricultural example.

Say you are interested in comparing the effectiveness of various fertilizers. Using the philosophy of vary only one circumstance at a time, you design your experiment, all at one location only, with only one cultivar, and all the experimental plots with the same soil type, and all production variables equal, like plant spacing. Does this sound like a good plan?

The other, questionnaire philosophy, leads you to an experiment spread at multiple locations, with different cultivars, and varying soil types, plant spacing and other production variables in a systematic way. This second experiment makes you learn, not only which fertilizer is best under some highly specific conditions, but how fertilizer effectiveness changes with varying conditions. It seems more useful.

• Sounds reasonable. I was expected something complicated and paradigm shifting. Apr 11 '20 at 19:44

Just to add to @kjetilbhalvorsen's answer, the insight Fisher got, or what came out of that is factorial design. This often cited excerpt comes from Arrangement of Field experiment, pg 10 (or 511). Though it seems maybe common sense to us now, Fisher notes most experiments done during that time involves single factors. In Terry Speed's introduction to this work, he notes:

So that the reader can better appreciate Fisher's remark about asking Nature few questions, it is worth quoting from Russell (1926, p.989, our italics): "A committee or an investigator considering a scheme of experiments should first and ask whether each experiment or question is framed in such a way that a definite answer can be given. The chief requirement is simplicity: only one question should be asked at a time."

Fisher proposed a field experiment design that can understand the effect of early/late addition of fertilizer containing sulphate or muraite, and other factors using only 96 plots as compared to using 216 plots if we are to conduct the sulphate/muraite experiments separately.

Of note is that factorial experiments actually opened the path for complex experimental design and also analysis of confounding effects.