In my experience sensitivity analysis in statistical settings generally means some variation on: I estimated the model again, by altering one or more researcher-selected parameters, in order to understand how the assumptions enacted by these choices affect my results.
Example from analysis with missing data
Suppose one has a dichotomous (0/1) variable in one's model, and some of its values are missing. Perhaps you used analysis of only complete cases, or some imputation strategy to address this. You might also add a sensitivity analysis whereby you:
- replace all missing values with a 0, and re-estimate your model, and
- replace all missing values with a 1, and re-estimate your model.
The results of this sensitivity analysis might in a simple case bracket the largest effect data missingness can have on the results of your analysis strategy.
Example from analysis with uncertainties and biases
In research on rates of homicide of transgender people in the US, I was presented with overlapping biases towards under-count in the numerator, and uncertainty about the denominator for homicide rates, and proceeded to estimate homicide rates by:
- assuming no under-count, and assuming under-count at rates 0.2, 0.5, and 0.8 to create my numerators, and
- using (and interpreting) three different prevalence estimates from the wider literature to create my denominators.
With twelve estimates per homicide rate, I was able to note which qualitative patterns depended strongly on these assumptions, and which depended weakly or not at all on the assumptions.
Speaking to sensitivity analyses on Bayesian priors
There are distributions of prior belief, including distributions in expert opinion, and previous published results. One may repeat one's analysis using values at the extremes of these distributions, and at the center, or aligned with the centers of modes of these distributions to explore the consequences for model results given different researcher assumptions.
What do with the results of such a sensitivity analysis?
- Combine the results as a model ensemble; describe the distribution of results
- Examine which results are qualitatively or quantitatively robust to varying assumptions, and which are strongly dependent on varying assumptions. The latter may indicate that the nature of that which you study varies depending on which assumption holds true, or it may indicate poor prior knowledge, which is important to nail down firmly to get valid model results.