What is Response Surface Methodology (RSM)?

Question

Q1: In layman terms (hopefully still accurate/correct), and in the light of my attempt below, what is RSM?

Q2: Same question as above, but without "in the light of my attempt below". You are free to explain RSM in layman terms in the perspective that you think is most healthy.

Q3: Can I say that Artificial Neural Networks (or its variants, like Recurrent Neural Networks [or its variants like LSTM]) are just special-case implementations of RSM?

My attempt (includes more questions):

I am reading this: http://www.stat.ufl.edu/personnel/usrpages/RSM-Wiley.pdf

I found this equation:

$$ y = f'(\mathbf{x})\beta + \epsilon $$ where:

• $y$ is target classification/regression label (they call it response of interest; is there any difference in calling it my way or their way?).
• $\mathbf{x}$ is a $k$ dimensional vector (they say $(x_1,x_2,\ldots,x_k)'$; I guess $'$ means transpose to say that it's a column matrix? and it's only a common notational convention that's all? Am I right?)
• $f(\mathbf{x})$ they say it's a vector function of $p$ elements -- what does this even mean?:
  • Is this a notational abuse? Did they mean $f$ instead of $f(\mathbf{x})$? Because in my understanding $f$ is a function and $f(\mathbf{x})$ is the value of function $f$ when given input $\mathbf{x}$.
  • Does vector function simply mean that it's a function that its co-domain is a $p$-dimensional vector?
• $\beta$ is a $p$-dimensional vector too (they say a vector of $p$ unknown constant coefficient referred to as parameters).
• $\epsilon$ is some error term that is believed to have a mean of $0$ (do we need to believe that it's mean is $0$?)
• I guess $f'(\mathbf{x})$ means a transposed vector? Am I right? Therefore $f'(\mathbf{x})\beta$ is essentially a multiplication of a $p \times 1$ matrix against a $1 \times p$ matrix? So the output is a $p \times p$ matrix?
• If $f'(\mathbf{x})\beta$ is a $p \times p$ matrix, then $$f'(\mathbf{x})\beta + \epsilon = f'(\mathbf{x})\beta + \begin{bmatrix}\epsilon&0&0&\ldots\\0&\epsilon&0&\ldots\\\vdots\\0&0&0&\ldots&\epsilon\end{bmatrix} $$ Am I right?

If all is good, then $y$ is a $p \times p$ matrix! I don't understand why this is helpful. Did I make an error somewhere? Or is it that $y$ is modeled as a $p \times p$ matrix?

Answer 1

Just from a cursory reading, and not knowing much about experimental design, RSM just seems to be the idea that you assume that the variables which you can adjust are related to the output via a function of some form, and you fit this function by doing experiments. It is the same sort of general model as you consider when fitting a preidctive model (like a neural network), but in RSM you are assuming that you can adjust the $x$-variables, which is not generally the case when fitting a predictive model.

There is a book on experimental design with a really nice example showing how to use this idea to design a toy helicopter. This pdf seems to have a similar example. It might be a good idea to get a look at the example in order to understand what they are trying to do.

• You can call $y$ anything you want
• Yes, it means transpose. It appears that they care whether a vector is a row or a column, and "vector" means column by default.
• Yes, vector function means that it is a function which takes a $k$-dimensional (column) vector to a $p$-dimensional (column) vector. The notation $f(\mathbf{x})$ does technically mean the value of the function $f$ at $\mathbf{x}$, but it is very common to mix up the two notations. For example, people might say "the derivative of $\sin(x)$" instead of "the derivative of the sine function."
• $\beta$ is a $p$-dimensional column vector.
• You could have an error term whose mean wasn't zero, but then you could include that part of the error term into $f$, so assuming the mean is zero doesn't make the model any less general. See also: linear regression.
• Yes, $f'(\mathbf{x})$ means the transpose. But since $f(\mathbf{x})$ is a column, $f'(\mathbf{x})$ is a row (a $1 \times p$ vector) and $f'(\mathbf{x})\beta$ is matrix multiplication of a $1 \times p$ and a $p \times 1$, which results in a scalar, not a $p \times p$ matrix.
• No, $y$ is a scalar. The only place where you made a mistake is in the last step. Look at the examples that appear later in the same page, and you will see that $y$ is not supposed to be a matrix.