Without going into the details of a statistical simulation that I am working on, I would like to ask for advice for the following problem.
I am simulating the mean sqaured error (MSE) of a set estimators under many conditions, most of which can be chosen on continuous scales, which are constrained to the $[0,1]$ interval (e.g. correlations between variables in the simulation). For different discrete values (say $[0,0.1,0.2,...,1]$ for each variable in the simulation) and their fully crossed combinations I simulate and obtain the MSE of all estimators.
The results of such simulations are easy to interpret and illustrate (e.g. visualize) as long as we are dealing with a small number of conditions (say, one or two). An obvious approach is to plot contuors of MSE for the levels of conditions. For higher numbers of conditions, we have an essential problem which is akin to multivariate data analysis with one predicted variable (MSE) and multiple predictors (conditions) in an experimental setting.
I am in the situation that I want to compare the results (MSE) under a moderately large number of conditions (say, four or five) across a number of different estimators (say, ten). What are useful ways to describe such results, e.g. in academic papers? Clearly it is impossible to report MSE for all combinations of conditions and estimators, nor are visual plots straight forward. How can I discover patterns in my simulation results?
Two thoughts of mine on this problem are:
- Use MSE as a predicted variable and the conditions as predictors with interactions in a regression model. Interpret coefficients as sensitivity of condition to change the MSE.
- Estimate the "variance of MSE" between conditions and across all conditions, plus mean MSE across all conditions. Larger variance (or standard deviation) of MSE across conditions indicates that under some conditions MSE is 'high'.
I'd be also interested in visual approaches or papers discussing presentation of simulation studies.