3
$\begingroup$

I have a physical system which takes a number of inputs $x_i$ and produces an output $error$.

$$ Y = f(x_1, x_2, x_3, .. x_{1000}) $$

The function $f()$ can be evaluated by running a compute-intensive simulation of a model.

I want to find the $x_i$ to which $Y$ is most sensitive. In practice, I want to optimize the values for the few input variable which would give maximum return (in terms improving the system performance).

I can think of randomly changing each of the $x_i$ by a small amount around the existing value and record the output. Repeat the experiment by few hundred times and compute the correlation between $x_i$ to $Y$ and pick the inputs with high correlation.

I am wondering if there is a more formal method to achieve this.

One important constraint in my particular problem is that each model evaluation requires a computationally intensive simulation of about 10 minutes and $x_i$ is of size $1000$ to $2000$.

|cite|improve this question
$\endgroup$
1
$\begingroup$

I think that you could look into the field of Sensitivity Analysis : https://en.wikipedia.org/wiki/Sensitivity_analysis. In your case, I would advise to compute the Sobol' indices (https://en.wikipedia.org/wiki/Variance-based_sensitivity_analysis).

These indices represent the fraction of variance carried by a variable and/or a set of variables. Several R packages exist in order to compute first and second order indices quite efficiently, by using specific designs.

In your case, as the number of model evaluation is pretty small and the number of inputs is large, you could try to look into surrogate based sensitivty analysis (see for instance https://doi.org/10.1016/j.apm.2013.01.019): Take a well behaved initial design (Latin Hypersquare or other space filling designs), and based on these evaluations, build a surrogate model (using Kriging). This surrogate will then be used for intensive computations, and can give some insightful results.

Be aware however that due to the high number of inputs, an accurate surrogate will probably need a lot of initial runs to be generated. A usual rule of thumb is to take $10d$ initial design points, where $d$ is the input dimension.

|cite|improve this answer
$\endgroup$
  • $\begingroup$ Thanks. Is there any particular method of sensitivity analysis would you suggest in cases where the number of model inputs is large (~2000) but we can afford to simulate only few hundred iterations? $\endgroup$ – Suresh Apr 5 '18 at 11:54
  • 1
    $\begingroup$ I edited my answer to adress your comment $\endgroup$ – V. Traboulistopopoulos Apr 5 '18 at 12:21
  • $\begingroup$ In my case, Even though I have 1000 input variables, I am only interested in the only 5 top most influential variables. Will this relaxed requirement in any way help to reduce the number of evaluations needed ( I am using sobol so far and have used a heuristic to pick only 100 variables out of 1000). Any suggestions/pointers would be helpful. Thanks. $\endgroup$ – Suresh Apr 9 '18 at 17:16
  • 1
    $\begingroup$ I think that the problem lies in the fact that we don't know beforehand which 5 variables will be the most influent on the output. Sobol' indices would be a way to "rank" the input variables with respect to their influence. You could maybe try to use a sparse grid at first, to have a first insight on the variable with high influence, and then select those variables to make a more deep sensitivity analysis ? $\endgroup$ – V. Traboulistopopoulos Apr 10 '18 at 7:22
6
$\begingroup$

The function is basically a surface in your 100 dimensional space or so. And at any given point in the space, it's more sensitive to the inputs for which it has the highest value on the partial derivatives for that input.

That means, your function may be sensitive to one input at one point in your hight dimensional space, and sensitive to another input in another place in your space.

Mathematically speaking, you should calculate the partial derivatives of your function for each input, and then analyze those derivatives to see which one is usually higher than the others, if that's what you're interested in.

In other words, it may be the case that your function is very sensitive to $x_1$ if $x_2 = 5$, and extremely sensitive to $x_{10}$ if $x_2 = 100$.

You can calculate those derivatives either analytically if you have the closed form of the function, or estimate those derivatives at randomly given input points using numerical algorithms.

In practice, you usually don't have the analytical form of the function. What you can do is to randomly choose from your data set (an iid sample), and numerically estimate the derivatives for each input feature. With a large enough sample, you get a nice representation of the distribution of those partial derivatives, and then you can decide what to do with them depending on your calculated value sets (one set per input feature).

It may very well be the case that for some you get a nice gaussian distribution, for some you may see white noise, and for some you see a rather uniform distribution, etc. It really depends on your data AND your function.

|cite|improve this answer
$\endgroup$
  • $\begingroup$ In my case, I don't have a means to compute the partial derivatives. I can only compute it numerically by doing a delta change to $x_i$. I will update the question with more details. $\endgroup$ – Suresh Apr 3 '18 at 11:08
  • 1
    $\begingroup$ There is a third way to compute derivatives in addition to analytic/symbolic computation and finite approximation à la $\frac{f(x+h)-f(x)}{h}$: automatic differentiation. $\endgroup$ – Federico Poloni Apr 3 '18 at 14:30
  • 1
    $\begingroup$ yes, and this is what I consider as a numerical solution. $\endgroup$ – adrin Apr 3 '18 at 14:33
2
$\begingroup$

If your output is linearly related to the inputs then, you can try the Regression Analysis in context of sensitivity. We can calculate the standardized beta coefficient as the direct measure of sensitivity. because in the calculation of standardized beta coefficients, every input is firstly standardized w.r.t mean and standard deviation. and it is very simple to interpret.

for reference, https://www.listendata.com/2015/04/standardized-vs-unstandardized.html https://en.wikipedia.org/wiki/Standardized_coefficient

|cite|improve this answer
$\endgroup$
  • 1
    $\begingroup$ The prospect of optimizing the response either precludes a global linear relationship (for which there are no extrema) or implies constraints, both of which militate against applying your approach. $\endgroup$ – whuber♦ Apr 3 '18 at 14:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.