Q1: In layman terms (hopefully still accurate/correct), and in the light of my attempt below, what is RSM?

Q2: Same question as above, but without "in the light of my attempt below". You are free to explain RSM in layman terms in the perspective that you think is most healthy.

Q3: Can I say that Artificial Neural Networks (or its variants, like Recurrent Neural Networks [or its variants like LSTM]) are just special-case implementations of RSM?

My attempt (includes more questions):

I am reading this: http://www.stat.ufl.edu/personnel/usrpages/RSM-Wiley.pdf

I found this equation:

$$ y = f'(\mathbf{x})\beta + \epsilon $$ where:

  • $y$ is target classification/regression label (they call it response of interest; is there any difference in calling it my way or their way?).
  • $\mathbf{x}$ is a $k$ dimensional vector (they say $(x_1,x_2,\ldots,x_k)'$; I guess $'$ means transpose to say that it's a column matrix? and it's only a common notational convention that's all? Am I right?)
  • $f(\mathbf{x})$ they say it's a vector function of $p$ elements -- what does this even mean?:
    • Is this a notational abuse? Did they mean $f$ instead of $f(\mathbf{x})$? Because in my understanding $f$ is a function and $f(\mathbf{x})$ is the value of function $f$ when given input $\mathbf{x}$.
    • Does vector function simply mean that it's a function that its co-domain is a $p$-dimensional vector?
  • $\beta$ is a $p$-dimensional vector too (they say a vector of $p$ unknown constant coefficient referred to as parameters).
  • $\epsilon$ is some error term that is believed to have a mean of $0$ (do we need to believe that it's mean is $0$?)
  • I guess $f'(\mathbf{x})$ means a transposed vector? Am I right? Therefore $f'(\mathbf{x})\beta$ is essentially a multiplication of a $p \times 1$ matrix against a $1 \times p$ matrix? So the output is a $p \times p$ matrix?
  • If $f'(\mathbf{x})\beta$ is a $p \times p$ matrix, then $$f'(\mathbf{x})\beta + \epsilon = f'(\mathbf{x})\beta + \begin{bmatrix}\epsilon&0&0&\ldots\\0&\epsilon&0&\ldots\\\vdots\\0&0&0&\ldots&\epsilon\end{bmatrix} $$ Am I right?

If all is good, then $y$ is a $p \times p$ matrix! I don't understand why this is helpful. Did I make an error somewhere? Or is it that $y$ is modeled as a $p \times p$ matrix?


Just from a cursory reading, and not knowing much about experimental design, RSM just seems to be the idea that you assume that the variables which you can adjust are related to the output via a function of some form, and you fit this function by doing experiments. It is the same sort of general model as you consider when fitting a preidctive model (like a neural network), but in RSM you are assuming that you can adjust the $x$-variables, which is not generally the case when fitting a predictive model.

There is a book on experimental design with a really nice example showing how to use this idea to design a toy helicopter. This pdf seems to have a similar example. It might be a good idea to get a look at the example in order to understand what they are trying to do.

  • You can call $y$ anything you want
  • Yes, it means transpose. It appears that they care whether a vector is a row or a column, and "vector" means column by default.
  • Yes, vector function means that it is a function which takes a $k$-dimensional (column) vector to a $p$-dimensional (column) vector. The notation $f(\mathbf{x})$ does technically mean the value of the function $f$ at $\mathbf{x}$, but it is very common to mix up the two notations. For example, people might say "the derivative of $\sin(x)$" instead of "the derivative of the sine function."
  • $\beta$ is a $p$-dimensional column vector.
  • You could have an error term whose mean wasn't zero, but then you could include that part of the error term into $f$, so assuming the mean is zero doesn't make the model any less general. See also: linear regression.
  • Yes, $f'(\mathbf{x})$ means the transpose. But since $f(\mathbf{x})$ is a column, $f'(\mathbf{x})$ is a row (a $1 \times p$ vector) and $f'(\mathbf{x})\beta$ is matrix multiplication of a $1 \times p$ and a $p \times 1$, which results in a scalar, not a $p \times p$ matrix.
  • No, $y$ is a scalar. The only place where you made a mistake is in the last step. Look at the examples that appear later in the same page, and you will see that $y$ is not supposed to be a matrix.
  • $\begingroup$ Thank you very much dude. But this looks to me like a special case of Artificial Neural Network stuff. Right? Looks like we are trying to find the program that generated the sequence that we are observing, and model this program as a linear function. No? $\endgroup$ – caveman Apr 20 '16 at 0:01
  • $\begingroup$ One more question I forgot: does the note on error $\epsilon$ mean that every prediction error is, by definition, 1) additive, and 2) with a mean of 0? $\endgroup$ – caveman Apr 20 '16 at 0:09
  • 2
    $\begingroup$ It seems like that's not quite what RSM is doing. They don't really care about whether the model is correct, just whether the directions of increase/decrease are correct. This is because they don't want an accurate model; they just want one that tells them how to change $\mathbb{x}$ to make $y$ bigger. $\endgroup$ – Flounderer Apr 20 '16 at 1:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.