I would like to create a second order polynomial model using Response Surface Methodology (RSM) for a non-polynomial mathematical model. For example, I would like to represent $f(x)=x_1 + \sin(x_1x_2) - x_2x_3$ in a polynomial using RSM. Is there any software I could use to do that? I tried using MATLAB but I could not figure out which function to use.
Have a look at the rsm package in R and the companion paper in 2009 Journal of Statistical Software by its author, Russell Lenth.
The following MATLAB functions come with neither recommendation (yea or nay) or experience from me in these particular tools. I think you can choose from among them depending on your preference as to "how to do business".
There are several tools you can use in MATLAB:
Or you can just form the linear and quadratic terms yourself and use linear least squares.
If you want to venture into the nonlinear parameter domain, which you don't need to do for a quadratic response surface model (but I'm not sure how much success you will have with quadratic RSM - I don't know what your intended use is, but quadratic RSM might not be a very good thing, and will be very lousy for modeling trig functions over a very large domain. Maybe you should just use a Taylor series approximation, paying attention to the remainder term.).