-1
$\begingroup$

I was asked to evaluate how the output changes in response to input variables. For this I have randomly create sensible value of input variables. These are then imported into a software which yields the output variables, however I do not know which formula is used for the calculations. How can I make statements about the sensitivity? Is that possible at all? What are my alternatives?

EDIT:

I am working on evaluating the cost/benefit ratio of some safety measure against natural hazards. Hereby I use different input values depending on which objects (house, bridge, field, etc.) have been affected by which natural hazard (Water, snow, rock, etc).

I have simulated input value ranges for a a certain object being hit by a certain process - ex. family house hit by avalanche. Then I import these into the software (not important which) and it yield the output values by using complex algorithm. The output values is the cost/benefit ratio. The (not important which) software offers a standard cost/benefit ratio calculated using standard input values. I then calculate the difference between the standard cost/benefit ratio with generated one by me. The output look like:

enter image description here

The mean median value in this case is 0.0. But say it was negative, can I say that the standard value generally overestimates the cost/benefit ratio?

More importantly: What can I say about sensitivity from this? I would like to evaluate if some processes and object types are more sensitive to changes in input values than other - is looking at the variabilty of the differences enough?

Until now I only have graphs in my analysis, I would like to get some numbers, statistical test, what can I use?

Additionaly, there is a treshold value for the cost/benefit ratio which indicates the economic efficiency for a certain project(=construction of the safety measure). How can I identify the amount of accepted variability in the input values before the project goes from economically efficient to inefficient ( or vice versa)? Would it be sensible to run a (logistic ?) regression such as cost/benefit ratio ~ Natural hazard + object type + input value?

Very important detail, I use R to do the statistics.

$\endgroup$
  • $\begingroup$ I suppose if you simulated the combinations of input value ranges that could give you what you want. That could get messy with a lot in input values and a closed UI for the software. Any more detail to share? Software package, input variables, documentation? $\endgroup$ – Simon Apr 24 '17 at 7:50
  • $\begingroup$ I don't follow the last part about the threshold value and logistic regression. Perhaps ask this as a separate question. $\endgroup$ – Simon Apr 24 '17 at 12:27
2
$\begingroup$

Using R, the src function in the sensitivity package may serve you well. If you can assume the blackbox is a linear model, then this should be enough.
This function can be run with a matrix of parameters and an output vector. The documentation has an easy example to follow.

Otherwise, in this post the OP tried to incorporate a blackbox calculation into the sobol function (source code here).

In terms of testing the difference between the model and your hand calculations, you may be able to do a t-test or Wilcoxon rank sum test

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.