2
$\begingroup$

I have few questions regarding the Morris method (as decribed e.g. in Campolongo, Cariboni, Saltelli, Environmental Modelling & Software 22, 2007 or Wenthworth et al. J. Uncertainty Quantification 2016) a global sensitivity method. I describe it below in some details as it is not very widely known.

Method definition

It uses the notation of the elemenatry effects defined as follows $$d_i(q) = \frac{f(q_1,\dots,q_{i-1},q_i+\Delta,q_{i+1},\dots,q_p)-f(q)}{\Delta} = \frac{f(q+e_i)-f(q)}{\Delta}$$ where $\Delta$ is the step size chosen from the set $\Delta \in \{ \frac1{l-1}, \dots, 1-\frac1{l-1} \}$. Note that it assumes parameters mapped to $[0, 1]^p$.

Constructed in this way, $d_i$ quantifies the approximate, large scale, local sensitivity of the model response relative to the ith parameter. The following settings are recommended: $l$ should be even and $\Delta = \frac{l}{2(l-1)}$ which should garantee equal probability sampling from the distribution.

$$d_i^k(q) = \frac{f(q^k+\Delta e_i)-f(q^k)}{\Delta}$$ is the elementary effect associated with $i$th parameter and kth sample. With $r$ sample points per parameter, the Morris indices for the parameter $q_i$ are $$\mu_i^* = \frac1{r}\sum_{k=1}^{r} |d_i^k|, $$ $$\sigma_i^2 = \frac1{r-1} \sum_{k=1}^{r}(d_i^r - \mu)^2, \mbox{where } \mu = \frac1{r}\sum_{k=1}^{r} d_i^k.$$ The mean quantifies the individual effect of the input on output, whereas the variance incorporates variability due to parameter interactions. Ranking of the parameters is done using $\sqrt{\mu_i^{*2} + \sigma_i^2}$.

Screening algorithm (Wentworth et al. 2016)

  1. Create a $(p+1)p$ matrix $A$ with ones in the lower triangle such that $$A = \begin{bmatrix} 0 & 0 & \dots & 0 \\ 1 & 0 & \dots & 0 \\ \dots & & \ddots & \\ 1 & 1 & \dots & 1 \end{bmatrix}$$
  2. Choose the step size $\Delta$ as above equal $\frac{l}{2(l-1)}$.
  3. Select starting vector $q^* \in \mathbb{R}^{1\times p}$.
  4. Construct a $p\times p$ diagonal matrix $D^*$, whose entries are randomly chosed from $\{ -1,1\}$.
  5. Calculate the sampling matrix $$ A_s = J_{p+1,1}q^* + \frac\Delta2 [(2A - J_{p+1,p})D^* + J_{p+1,p}]P^*,$$ where $J_{i,j}$ is a $i\times j$ matrix with all ones and $P^*$ is a permutation of the identity matrix.
  6. For parameters defined not in unit hypercube, i.e. $q\in[l_i,u_i]$ for $i=1,\dots,p$, take $l=[l_1,l_2,\dots,l_p]$ and $u=[u_1,u_2,\dots,u_p]$. The sampling matrix is then scaled to match the range of parameters $$ C=J_{p+1,1}l + A_s(D(u-l)), \;\;\;\;\;\; (*)$$ where $D(u−l)$ is a diagonal matrix with entries $u−l$.
  7. Compute the elementary effects for $i=1,\dots,p.$ Let $C_k$ denote the $k$th row of $C$. Then $$ d_i = \frac{f(C_m) - f(C_n)}{\Delta}, \;\;\;\;\;\; (**)$$ where $m$ and $n$ denote the indices such that $m$th row and $n$th row differ in the $i$th entry.
  8. Repeat steps 1-7 for $r$ samples. The Morris statistics, $\mu_i^*$ and $\sigma_i^2$, are calculated as described in the Method section by taking the average of the local elementary effect.

Note 1 from the author: denominator of (**) in step 7 is $\Delta$ for all $q_i, i = 1,\dots, p$. The elementary effects must be computed using the scaled step size, even though model responses are computed at the parameter values, which are mapped using (*).

Note 2: Wentworth et al. and others suggest $l=20$ and $r=50$.

Discussion/Questions

I have trouble to see why $\Delta$ is choosen to be $\frac{l}{2(l-1)}$, which for $p=20$ is equal $10/19$. When selecting $q^*$ as in point 3 of the algorithm, the $q_i$ of interest has to be selected from the set $\{0,1/(l-1),2/(l-1),...,1-\Delta\}$ (according to original Morris 1991 paper, otherwise it might be that after adding delta its value is larger then 1.). I.e. $q_i$ will assume values from 0 to 9/19.

Question 1: Why not simply generate sequence between 0 and 1 with elements equidistantly spaced with step $\frac{1}{(l-1)}$? It would be analog to the above scenario.

Question 2: Would it make better sense to calculate, instead of an elemenatry effect, a local normalized sensitivity index similar to how it is done in the Matlab SimBiology toolbox, see the 'full' normalization case (of course with $\Delta$ instead of differentials $\partial x$ or $\partial k$ and without the time dependency): https://uk.mathworks.com/help/simbio/ref/normalization.html It would have the advantage that it is dimensionless compared to the elementary effects.

$\endgroup$
1
$\begingroup$

I'm "answering" this question because I stumbled upon it while looking for the same question.

The first question: why is $$\Delta = \frac{l}{2(l-1)}$$ (your $l$, usually $p$)? The idea is that $\Delta$ is the actual jump step when your inputs are in $[0, 1]$. However, it is easier to think of index-jumping in the array of possible inputs, where the jump is $l/2$, which is identical to the above definition of $\Delta$. When you jump half the array forward or backward (and the array has an even number of steps $l$) there are always exactly two pairs of indices that can be jumped to or from. So in your example, if you number the indices starting with 0: $\{0, 1, 2, \dots , 19\}$. From 0 you always jump to 10, from 1 to 11, ... and the same is true for jumps back (from 10 to 0, from 11 to 1, ...). This ensures that (given a randomly chosen start point) each element of the input array is chosen with the same probability, therefore giving you uniform coverage of the input space. There is a nice graphical explanation of this in Saltelli et al. (2008, p. 117ff).

I cannot answer question 2.

Saltelli et al (2008, p. 120) argue no, because in a model of $y = x_1 + x_2$ where $x_1 \in [0, 1]$ and $x_2 \in [0, 10]$ the second factor seems more important. However, if $\Delta$ is normalized (different for each $x_i$), both factors would be equally important. This points towards not scaling and seems sensible to me.

Monari and Strachan (2017) argue for scaling:

The sensitivity indexes employed in the Morris method can be dependent on the variation ranges applied. In partic ular, leaving the variations applied to each parameter with its own units can produce misleading results since the calculated elementary affects will have in turn different units, making it impossible to compare them. To avoid this issue model input samples have been scaled and centred so that each of them has mean equal to zero and standard deviation equal to one.

So do Sin and Gernaey (2009):

This refinement consists of scaling the elementary effects to obtain a non-dimensional measure. This is needed for two particular reasons: (1) to prevent any misleading conclusions about the importance of the factors in complicated models and (2) to reliably compare the results of the analysis on different model outputs. This necessity arises especially when a model contains outputs and factors with large differences in their values/units, for example one or more orders of magnitude.

This also sounds logical to me. So which is better, scaling or not scaling? Note that scaling the input factors $q_i$ or not makes no difference, since they only appear as arguments to the output function $f$ and therefore have no influence on the actual elementary effects. No scaling means $\Delta = \frac{l}{2(l-1)}$, scaling means having $\Delta_i$ and using them depending on which $q_i$ is currently changed.

I have no idea.

Scaling $f$ (like Sin and Gernaey 2009) is an additional issue and in my opinion can only increase comparability if one is looking at multiple outputs $f$.


Saltelli, Andrea, Marco Ratto, Terry Andres, Francesca Campolongo, Jessica Cariboni, Debora Gatelli, Michaela Saisana, and Stefano Tarantola (2008); Global sensitivity analysis: The primer, John Wiley & Sons, doi:10.1002/9780470725184

Monari, F. and Strachan, P. (2017); Characterization of an airflow network model by sensitivity analysis: parameter screening, fixing, prioritizing and mapping; Journal of Building Performance Simulation 10, p. 17 - 36, doi: 10.1080/19401493.2015.1110621

Sin, Gürkan and Krist V. Gernaey (2009); Improving the Morris method for sensitivity analysis by scaling the elementary effects, Computer Aided Chemical Engineering 26, p. 925 - 930, doi: 10.1016/S1570-7946(09)70154-3

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.