Comparing many performance curves: request for data visualization tips

Question

My request

I wish to compare many performance curves. Plotting all the curves (with error bars) in a single figure makes a mess of things. I'm interested in ways to de-clutter my plot and make it easier to compare the curves.

This is not intended as a programming question: it's not necessary to provide code (although you can if you want to), a verbal description on how to improve the plot is enough.

Background information

My reviewers clamoured for a comparison of my new found analysis method to a variety of well known alternatives. So I'm running data simulations; lots of them. These simulations generate artificial data, based on a multitude of parameters that can be tweaked, and each method is applied to the dataset. The purpose of the study is to show how each analysis method behaves in response to a change in parameters.

I would pick a parameter and start changing it. For each value of the parameter, I run the simulation about 100 times, producing 100 data sets. Then I run each analysis method on each dataset, producing for value of the parameter and each method, a mean and standard deviation across the 100 runs.

The Python code below produces a toy example of the data I wish to visualize:

import numpy as np
from matplotlib import pyplot as plt

method_names = ['methodA', 'methodB', 'methodC', 'methodD', 'methodE']
parameter_values = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

# Mean performance of each method for each value of the parameter. Each
# column corresponds to a method, each row to a value of the parameter.
means = np.array([
  [ 0.33310882,  0.55161232,  0.71036095,  0.25674653,  0.69863089],
  [ 0.19724624,  0.61167882,  0.6102655 ,  0.30949569,  0.58623639],
  [ 0.1356461 ,  0.63687691,  0.56813548,  0.31290411,  0.52985315],
  [ 0.10735517,  0.63363713,  0.51832115,  0.3246267 ,  0.4784114 ],
  [ 0.08432418,  0.64023433,  0.48225627,  0.35112391,  0.43079314],
  [ 0.08762582,  0.63364727,  0.43214314,  0.34367382,  0.36650684],
  [ 0.08586268,  0.63693999,  0.43351215,  0.33518338,  0.34467524],
  [ 0.0741298 ,  0.64564111,  0.40943309,  0.36357895,  0.312961  ],
  [ 0.06163042,  0.62847129,  0.41779745,  0.36114122,  0.34724645],
  [ 0.07159902,  0.63879868,  0.38652708,  0.366425  ,  0.28765962]
])

# Standard deviation of the performance of each method for each value of the
# parameter. Each column corresponds to a method, each row to a value of the
# parameter.
stds = np.array([
  [ 0.11254176,  0.10631446,  0.06812396,  0.08699054,  0.06980061],
  [ 0.08628651,  0.10833594,  0.09483841,  0.1183296 ,  0.1024852 ],
  [ 0.06817238,  0.10773644,  0.12192901,  0.1277693 ,  0.13846137],
  [ 0.06689446,  0.10816033,  0.12069033,  0.11992669,  0.13071808],
  [ 0.05422928,  0.10254246,  0.12434407,  0.13343013,  0.1383579 ],
  [ 0.06296734,  0.1000487 ,  0.14946763,  0.13094066,  0.1616725 ],
  [ 0.06012606,  0.10337348,  0.13938654,  0.10372903,  0.16188025],
  [ 0.05196553,  0.10243771,  0.12804723,  0.12445235,  0.15411106],
  [ 0.04714007,  0.09093044,  0.14208883,  0.1209349 ,  0.16194828],
  [ 0.05830223,  0.1081157 ,  0.15168251,  0.12709928,  0.17751713]
])

And here is what a naive visualization might look like:

And these are only 5 curves... Í have about 15 curves to compare.

Sidenote

The obvious way to improve the plot is to show only a selection of the curves. I'm trying to focus on the most interesting ones in the paper, and banish the rest of them to the supplementary information section. The better I can make my plot, the least curves I would have to banish.

Answer 1

As was the recommendation in Color and line thickness recommendations for line plots, small multiples are a common solution for plots that have problems with overplotting. Here is an example with the 5 curves you provided.

It is a lot of information, but it is pretty easy to see that A, C & E are all decreasing. C & E have increasing variance for high parameter values, while B and D are fairly constant, and A hits the bottom (I would guess the metric has to be a positive value given the graphs.) Small multiples can extend to 15 curves, but when making the panels smaller IMO you should make the panels a bit more minimalist in the smaller space, e.g. ditch the gridlines, make the tick marks smaller and more sparse, etc.

Error bars make the overplotting problem even more problematic, so it is harder to stuff multiple error bars into one graph if the trajectories overlap. One alternative way I like though is to use semi-transparent areas as opposed to the points and bars. My labeling could use some work, but here is an example with these curves (plus and minus two standard deviations to make the areas a bit wider).

I had to put E in a separate panel, as it occluded A and C. Seeing the overlap of two areas is not that difficult given the right colors for the areas, three though is very difficult.