I'm evaluating different systems varying a certain parameter common to all of them. Let this parameter be
x. At each
x value I evaluate each system multiple times (eg: 10 times) and record the results. The issue is, with this number of evaluation (10 times) it's hard to get an idea about the underlying distribution of them. At some
x values they look uniformly distributed but at some
x values they look normally distributed. Apart from that in most cases there are outliers !!! And they also occur in one side ( like this - so it's skewed).
Different colors represent results for different systems. In each plot center line is the sample mean and the shaded area corresponds to $\pm2s$ (sample standard deviation). I would like to make a similar plot for these results as well. I have couple of ideas.
- Plot median as the center line and $Q1$ , $Q3$ as the shaded region. (optionally add outliers [ outside $3IQR$ from Q1 and Q3 ] like in box plots).
- Use a trimmed statistic (like 10% trimmed mean and standard deviation)
- Removing outliers and computing mean and SD as before (but I don't think it's a good idea).
- Fit a skewed distribution (like skew normal distribution). But not sure how I can represent it in the plot.
I'm planning to go with first option. Any suggestion is appreciated.