I have a vector of parameters $\theta=(n,ph,pt,\gamma)$ which are fed into a process which produces a result. I would like to quantitatively report the sensitivity of the result to the variation of a parameter(s). Univariate correlation isn't what I'm after because $\theta$ is sampled from a distribution. What do you think is my best bet? Let me know if more information is required.



The process is stochastic. Basically a person moves round a room touching surfaces (n times) stochastically, based on observed probabilities. By touching each surface they pick up an isotope with probability $\gamma$ . How much isotope they pick up depends on a variable pt, calculated experimentally.
After they finish touching surfaces, they wash their hands with probability and efficacy (ph).

Therefore the output is a scalar quantity of isotope and a vector of surfaces touched.


The isotope ($I$) on the person's hand is calculated in an additive manner after each surface contact so:

$$I=\lambda_i \, pt_i V_i$$

Where $V_i$ is the surface concentration of isotope. Let's assume we're dealing with one surface so $V$ is constant. Then the total sum of isotope on the person's hand after $n$ touches with the same surface would be:

$$I=\sum_{i=1}^n\lambda_i \, pt_i V_i$$

They then wash their hands with some efficacy $ph$. The left hand picture is a frequency density histogram of $I$. The right hand picture is a scatter graph between $I$ and $n$. enter image description here

However $n$ is drawn from a distribution of observed data. How can this be incorporated into sensitivity analysis?

  • $\begingroup$ There are numerous methods for sensitivity analysis among which some of the most popular are Morris' screening method, variance based sensitivity estimation (Sobol' method) and use of meta modelling. You should provide more information about your process to narrow the choice of tools. Is it deterministic or stochastic? Is it computer intensive? Are your input parameters correlated? Is your output a scalar or a vector? $\endgroup$ – Alfred M. Jul 11 '12 at 8:36
  • $\begingroup$ @AlfredM. Thank you for your input. To answer your queries I have added some info above. Does this help? $\endgroup$ – HCAI Jul 27 '12 at 19:51
  • $\begingroup$ The fact that it is stochastic complicates the matter substantially. How much time does it take to get an output for one $\theta$? The vector output of surface touched might be difficult to analyse, especially if, as I suppose, their size is random. But you may be able to derive a scalar output that makes sense out of it? If not, I suggest to first concentrate on the quantity of isotope and see what you can make of it. $\endgroup$ – Alfred M. Jul 28 '12 at 9:53
  • $\begingroup$ Agreed. As the major output variable, the quantity of isotope is generally of more interest. Generating a single run is very quick, less than a second. Do you recommend a particular method for analysing that with respect to n,ph,pt,lambda? $\endgroup$ – HCAI Jul 30 '12 at 6:40

The number of input variables is moderate as well as the time for a model evaluation. This is a favourable situation and detailed sensitivity indices can be obtained.

I would recommend to use the method of Sobol' with a quasi-Monte Carlo scheme.

Usually this method applies to deterministic model but as your model is cheap computationally wise, you can resort to some averaged approach. Particular case of handling stochastic models might exists in the recent literature but I do not know about them.

I assumed that your input variables are not correlated (note: that does not mean that do not interact inside the model).

One of the founding paper is

I. M. Sobol’ : Sensitivity estimates for nonlinear mathematical models, in Matem. Modelirovanie, 2 (1)(1990) 112 – 118. English Transl. : MMCE, 1(4), 1993.

but a more recent version (freely available on the Internet) is :

I. M. Sobol’ : Global sensitivity indices for nonlinear mathematical models and their Monte Carlo estimates. Mathematics and Computers in Simulation, 55:271–280, 2001

A recent assessment of the different techniques to estimate Sobol' indices is given by Saltelli et al. who gives clear guidelines :

A. Saltelli, P. Annoni, I. Azzini, F. Campolongo, M. Ratto et S. Tarantola : Variance based sensitivity analysis of model output. Design and estimator for the total sensitivity index. Computer Physics Communications, 181(2):259–270, 2010.

The primer by Saltelli et al. will allow you to dive more deeply into the subject.

Other methods could work but this one would be my first attempt in your situation. It is no too difficult to implement, quite robust and substantial literature and application cases are available.

The only thing that you should avoid is One at a time experience design. See

A. Saltelli et P. Annoni : How to avoid a perfunctory sensitivity analysis. Environ- mental Modelling & Software, 25(12):1508–1517, 2010.

for arguments supporting this claim.

The key-word for finding other methods and more details about the subject is global sensitivity analysis.

The case of the multivariate output can be dealt in several manners but it might required another questions with details about the structure of your multivariate output and what you want to explore.

You can have a look at this question for a presentation of other sensitivity analysis methods.

Good luck !

  • $\begingroup$ The references are indeed comprehensive, thank you. I have read them and some by Dimov. I added a clearer explanation of my model.. What do you think? Also in a development of this model an input parameter may be correlated to another making it non-linear, do you suggest anything there? Regards, $\endgroup$ – HCAI Aug 4 '12 at 21:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.