I have the following problem:
Given inputs $x$ ($n$-dimensional vector) of scalars, ordered integers and unordered integers (i.e., labels) and one or several outputs $y$, I would like to estimate:
- Which inputs explain the best the outputs.
- To what extent variations of one input imply variations of the outputs.
This is supposed to be related to uncertainty and sensitivity analysis, which are quite broad fields. Do you know of any methods/resources with an approach related to my problem?
You can try one of the tools provided here. That is matlab solutions, very nice code and modern methods. Firstly I would suggest you to try graphical tools from the library to make sense about the data.
As you did not provide the details on what you need here are some comments on the methods implied:
Global Sensitivity Analysis. Global sensitivity analysis is the study of how the uncertainty in the output of a model (numerical or otherwise) can be apportioned to different sources of uncertainty in the model input. Global could be an unnecessary specification here, were it not for the fact that most analysis met in the literature are local or one-factor-at-a-time.
Monte-Carlo (or Sample-based) Analysis. Monte Carlo (MC) analysis is based on performing multiple evaluations with randomly selected model input, and then using the results of these evaluations to determine both uncertainty in model predictions and apportioning to the input factors their contribution to this uncertainty. A MC analysis involves the selection of ranges and distributions for each input factor; generation of a sample from the ranges and distributions specified in the first step; evaluation of the model for each element of the sample; uncertainty analysis and sensitivity analysis.
Response Surface Methodology. This procedure is based on the development of a response surface approximation to the model under consideration. This approximation is then used as a surrogate for the original model in uncertainty and sensitivity analysis. The analysis involves the selection of ranges and distributions for each input factor, the development of an experimental design defining the combinations of factor values on which evaluate the model, evaluations of the model, construction of a response surface approximation to the original model, uncertainty analysis and sensitivity analysis.
Screening Designs. Factors screening may be useful as a first step when dealing with a model containing a large number of input factors (hundreds). By input factor we mean any quantity that can be changed in the model prior to its execution. This can be a model parameter, or an input variable, or a model scenario. Often, only a few of the input factors and groupings of factors, have a significant effect on the model output.
Local (Differential Analysis). Local SA investigates the impact of the input factors on the model locally, i.e. at some fixed point in the space of the input factors. Local SA is usually carried out by computing partial derivatives of the output functions with respect to the input variables (differential analysis). In order to compute the derivative numerically, the input parameters are varied within a small interval around a nominal value. The interval is not related to our degree of knowledge of the variables and is usually the same for all of the variables.
FORM-SORM. FORM and SORM are useful methods when the analyst is not interested in the magnitude of Y (and hence its potential variation) but in the probability of Y exceeding some critical value. The constraint (Y-Ycrit < 0) determines a hyper-surface in the space of the input factors, X. The minimum distance between some design point for X and the hyper-surface is the quantity of interest.
Good luck!
To address the first question, I suggest you have a look at canonical correlation analysis and to a more recent dimension reduction technique called sliced inverse regression. On the latter, see the initial paper by Ker Chau Li
Sliced inverse regression for dimension reduction (with discussion). Journal of the American Statistical Association, 86(414):316–327, 1991.
It is freely available on the Internet. The version with the (interesting) comments you might have to buy thought.
Some important parameters for the choice of a method in you situation are :
Also you mention a possible multivariate output. If you have a few of them that represent completely different things, just do several independent sensitivity analysis.
If they are higly correlated or functional then it also change the problematic a lot.
You should make all this points clear before going for a given methodology.
You may be able to use a variance-based global sensitivity analysis approach to answer the second question. According to Saltelli (2008), sensitivity analysis is
"...the study of how uncertainty in the output of a model can be apportioned to different sources of uncertainty in the model input..."
The approaches, such as those mentioned in the Saltelli book, normally focus on first generating an input sample, which is subsequently run through a model to generate outputs, and then analysed. The output metrics which result, such as the total sensitivity index $S_{t_i}$ and first-order sensitivity index $S_i$ represent the main effect contribution of each input factor to the variance of the output. Variance based approaches decompose the variance in the output. They are computationally demanding, and require a specific input sample.
For your purposes, given that you have an existing ranch of data is an alternative method such as that suggested by Delta Moment-Independent Measure (Borgonovo 2007, Plischke et al. 2013) and implemented in the Python library SALib.
The following code, taken from the example allows you to generate the sensitivity indices from your existing data:
from SALib.analyze import dgsm
from SALib.util import read_param_file
# Read the parameter range file
problem = read_param_file('../../SALib/test_functions/params/Ishigami.txt')
# Perform the sensitivity analysis using the model output
# Specify which column of the output file to analyze (zero-indexed)
Si = dgsm.analyze(problem, param_values, Y, conf_level=0.95, print_to_console=False)
# Returns a dictionary with keys 'vi', 'vi_std', 'dgsm', and 'dgsm_conf'
# e.g. Si['vi'] contains the sensitivity measure for each parameter, in
# the same order as the parameter file
