1
$\begingroup$

I'm reading about Design of Experiments via various textbooks (e.g. Montgomery's), and powerful global optimization methods (such as Ant Colony Optimization) are not used. They rely only on a basic sampling scheme in the search space; full factorial or otherwise, and then fit a linear model (perhaps with quadratic and interaction terms) of the experimental factors onto the response variable. Then they use that model to estimate optimal values for the inputs in order to maximize that response variable.

What is the purpose of limiting methods like this? Just deploying global search algorithms would result in a better optimization outcome.

EDIT:

I understand that optimization isn't the only (or main) purpose of DoE, but it's often part of the purpose. See for example chapter 11 of Montgomery's textbook titled Design and Analysis of Experiments, where optimization is the focus. I wanted to know why powerful search tools are not considered when optimization is a big part of the goal.

Improve this question
$\endgroup$
5
  • 1
    $\begingroup$ It is not clear what exactly do you mean and where would you like to use optimization. $\endgroup$
    – Tim ♦
    Mar 7 at 11:14
  • $\begingroup$ @Tim DoE's purpose is often (but not always) optimization. For example, chapter 11 of Montgomery's book "Design and Analysis of Experiments", titled "Response Surface Methods and Designs", is about fitting the various experimental factors to the target response variable. That model is then used to maximize the response variable subject to those factors. In all the DoE material I've studied, in the parts pertaining to optimization, nobody discusses more powerful search methods. I wanted to know why. $\endgroup$
    – questioner50
    Mar 7 at 11:21
  • 1
    $\begingroup$ Optimization methods are used for D-optimality, see for instance stats.stackexchange.com/questions/266405/… Is this what you have in mind? $\endgroup$
    – kjetil b halvorsen ♦
    Mar 7 at 11:37
  • $\begingroup$ But optimization is used like this, here you have a paper reviewing one of the possible approaches ieeexplore.ieee.org/document/8957442 $\endgroup$
    – Tim ♦
    Mar 7 at 11:42
  • 1
    $\begingroup$ @Tim That answers my confusion, thank you. I was just reading material that was a bit too introductory. $\endgroup$
    – questioner50
    Mar 7 at 12:24

1 Answer 1

Reset to default
1
$\begingroup$

You refer chapter 11 of Montgomery's textbook, with the title "Response Surface Methods and Other Approaches to Process Optimization". So, interpreting your question as why response surface methods do not use global optimization methods.

Global optimization methods, as studied in optimization theory, is doing optimization on some known mathematical function. Implicitly, one is assuming that evaluation of function values at arbitrary points is cheap.

In contrast, Response Surface Methods tries to optimize some real-world system, often an industrial process. There is no known mathematical function representing the system, and "evaluation of values" is not cheap, it might imply running the process under non-standard conditions for some time (days? hours?), costs of changing process parameters, maybe even costs of destroyed or reduced production.

So this is very different from optimization of a known mathematical function, in reality, one is simultaneously trying to build a model of the system, and optimize it, under the restriction that evaluation is costly! That points to some analogy with active learning, see Motivations for experiment design in statistical learning?.

Another keyword is evolutionary operation see Best DoE method to fit Gaussian Process Regressor

Improve this answer
$\endgroup$
4
  • $\begingroup$ Global optimization methods are used on real-word systems, though. For example the algorithms here: math.unice.fr/~dreyfuss/rapport_11-12_2.pdf -- they're commonly used in industry on unknown functions. So I'm not sure about the distinction you're drawing here? Moreover, the RSM seems to be making even more explicit claims on the functional form. Many global optimizers (e.g. PSO) are model-free in that they do not assume any knowledge of the function, whereas RSM is (for example) a linear model with quadratic terms which is a strong assertion about the function being studied. $\endgroup$
    – questioner50
    Mar 7 at 17:42
  • $\begingroup$ That document does not seem to contain examples of the type you mention. Can you give some example of such application? $\endgroup$
    – kjetil b halvorsen ♦
    Mar 8 at 2:04
  • $\begingroup$ For example, suppose we want to optimize online advertisement click-throughs. That's our response (Y). Our input factors (X) could be ad location, ad size, ad color scheme, among other categorical or real-valued variables. The task is to optimize f(X) = Y. An optimization algorithm such as PSO would be able to do that without any assumptions on the form of f(X). It is simply applying heuristics to search through X in order to maximize Y. RSM on the other hand is asserting that f(X) follows a linear first-order or second-order model, at least locally. $\endgroup$
    – questioner50
    Mar 9 at 10:24
  • $\begingroup$ Please add this new context as an edit to your question, as it might make for more interesting answers. And please, spell out PSO. $\endgroup$
    – kjetil b halvorsen ♦
    Mar 9 at 14:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.