# sensitivity analysis when only input and output values are available in R

I was asked to evaluate how the output changes in response to input variables. For this I have randomly create sensible value of input variables. These are then imported into a software which yields the output variables, however I do not know which formula is used for the calculations. How can I make statements about the sensitivity? Is that possible at all? What are my alternatives?

## EDIT:

I am working on evaluating the cost/benefit ratio of some safety measure against natural hazards. Hereby I use different input values depending on which objects (house, bridge, field, etc.) have been affected by which natural hazard (Water, snow, rock, etc).

I have simulated input value ranges for a a certain object being hit by a certain process - ex. family house hit by avalanche. Then I import these into the software (not important which) and it yield the output values by using complex algorithm. The output values is the cost/benefit ratio. The (not important which) software offers a standard cost/benefit ratio calculated using standard input values. I then calculate the difference between the standard cost/benefit ratio with generated one by me. The output look like:

The mean median value in this case is 0.0. But say it was negative, can I say that the standard value generally overestimates the cost/benefit ratio?

More importantly: What can I say about sensitivity from this? I would like to evaluate if some processes and object types are more sensitive to changes in input values than other - is looking at the variabilty of the differences enough?

Until now I only have graphs in my analysis, I would like to get some numbers, statistical test, what can I use?

Additionaly, there is a treshold value for the cost/benefit ratio which indicates the economic efficiency for a certain project(=construction of the safety measure). How can I identify the amount of accepted variability in the input values before the project goes from economically efficient to inefficient ( or vice versa)? Would it be sensible to run a (logistic ?) regression such as cost/benefit ratio ~ Natural hazard + object type + input value?

Very important detail, I use R to do the statistics.

• I suppose if you simulated the combinations of input value ranges that could give you what you want. That could get messy with a lot in input values and a closed UI for the software. Any more detail to share? Software package, input variables, documentation? Apr 24, 2017 at 7:50
• I don't follow the last part about the threshold value and logistic regression. Perhaps ask this as a separate question. Apr 24, 2017 at 12:27

## 1 Answer

Using R, the src function in the sensitivity package may serve you well. If you can assume the blackbox is a linear model, then this should be enough.
This function can be run with a matrix of parameters and an output vector. The documentation has an easy example to follow.

Otherwise, in this post the OP tried to incorporate a blackbox calculation into the sobol function (source code here).

In terms of testing the difference between the model and your hand calculations, you may be able to do a t-test or Wilcoxon rank sum test