0
$\begingroup$

In Myers, Montgomery, & Anderson-Cook, C. M. (2016)$^1$ the authors provide (p. 235) a general methodology to find new $x'$ coordinates in the path of steepest ascent based on constrained optimization of the form

\begin{equation} \beta_0 + \beta_1 x_1 + \dots + \beta_k x_k \\ \text{s.t.} \sum_{i=1}^k x_i ^2 - r^2 = 0 \end{equation}

On which $r$ is the distance from the origin in the cube or hypercube. The authors solve it using Lagrange multipliers

$$ \max L = \mathbf{b'}\mathbf{x} - 2\lambda (\mathbf{x}'\mathbf{x} - r^2) $$

Which they found with $\partial L/\partial x = 0$ to be

$$ x_j = \frac{\beta_j}{2\lambda} $$

However, later on, they provide an algorithm (p. 239) on which new coordinates can be found by computing the step size with $$ \Delta x_j = \frac{\beta_i}{\beta_i/\Delta x_i},\quad i \neq j $$

So that the new point in the path of steepest ascent is just $\text{base} + \Delta x_k$. Should I interpret that $\beta_i/\Delta x_i = 1/2\lambda$?

Furthermore, in the Engineering Statistics Handbook (Technical Appendix 5A), the solution is

$$ x^* = \rho\frac{b}{\lVert b \rVert} $$

On which (I interpret) $\rho$ is the same as $r$ (the distance from the origin).

Are all those solutions equivalent? I interpret this as that the length is less important than the direction (?).

$^1$Myers, R. H., Montgomery, D. C., & Anderson-Cook, C. M. (2016). Response surface methodology: process and product optimization using designed experiments. John Wiley & Sons.

$\endgroup$

1 Answer 1

0
$\begingroup$

Yes, they are all equivalent, in the sense that they all say that the direction of steepest ascent is proportional to the regression coefficients. Some of the notations are artifacts of the methods used to derive that basic result.

In practice, you would experiment along the path of steepest ascent at specified distances. The third result is the most direct formula for finding those points.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.