5
$\begingroup$

My request

I wish to compare many performance curves. Plotting all the curves (with error bars) in a single figure makes a mess of things. I'm interested in ways to de-clutter my plot and make it easier to compare the curves.

This is not intended as a programming question: it's not necessary to provide code (although you can if you want to), a verbal description on how to improve the plot is enough.

Background information

My reviewers clamoured for a comparison of my new found analysis method to a variety of well known alternatives. So I'm running data simulations; lots of them. These simulations generate artificial data, based on a multitude of parameters that can be tweaked, and each method is applied to the dataset. The purpose of the study is to show how each analysis method behaves in response to a change in parameters.

I would pick a parameter and start changing it. For each value of the parameter, I run the simulation about 100 times, producing 100 data sets. Then I run each analysis method on each dataset, producing for value of the parameter and each method, a mean and standard deviation across the 100 runs.

The Python code below produces a toy example of the data I wish to visualize:

import numpy as np
from matplotlib import pyplot as plt

method_names = ['methodA', 'methodB', 'methodC', 'methodD', 'methodE']
parameter_values = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

# Mean performance of each method for each value of the parameter. Each
# column corresponds to a method, each row to a value of the parameter.
means = np.array([
  [ 0.33310882,  0.55161232,  0.71036095,  0.25674653,  0.69863089],
  [ 0.19724624,  0.61167882,  0.6102655 ,  0.30949569,  0.58623639],
  [ 0.1356461 ,  0.63687691,  0.56813548,  0.31290411,  0.52985315],
  [ 0.10735517,  0.63363713,  0.51832115,  0.3246267 ,  0.4784114 ],
  [ 0.08432418,  0.64023433,  0.48225627,  0.35112391,  0.43079314],
  [ 0.08762582,  0.63364727,  0.43214314,  0.34367382,  0.36650684],
  [ 0.08586268,  0.63693999,  0.43351215,  0.33518338,  0.34467524],
  [ 0.0741298 ,  0.64564111,  0.40943309,  0.36357895,  0.312961  ],
  [ 0.06163042,  0.62847129,  0.41779745,  0.36114122,  0.34724645],
  [ 0.07159902,  0.63879868,  0.38652708,  0.366425  ,  0.28765962]
])

# Standard deviation of the performance of each method for each value of the
# parameter. Each column corresponds to a method, each row to a value of the
# parameter.
stds = np.array([
  [ 0.11254176,  0.10631446,  0.06812396,  0.08699054,  0.06980061],
  [ 0.08628651,  0.10833594,  0.09483841,  0.1183296 ,  0.1024852 ],
  [ 0.06817238,  0.10773644,  0.12192901,  0.1277693 ,  0.13846137],
  [ 0.06689446,  0.10816033,  0.12069033,  0.11992669,  0.13071808],
  [ 0.05422928,  0.10254246,  0.12434407,  0.13343013,  0.1383579 ],
  [ 0.06296734,  0.1000487 ,  0.14946763,  0.13094066,  0.1616725 ],
  [ 0.06012606,  0.10337348,  0.13938654,  0.10372903,  0.16188025],
  [ 0.05196553,  0.10243771,  0.12804723,  0.12445235,  0.15411106],
  [ 0.04714007,  0.09093044,  0.14208883,  0.1209349 ,  0.16194828],
  [ 0.05830223,  0.1081157 ,  0.15168251,  0.12709928,  0.17751713]
])

And here is what a naive visualization might look like:

Example visualization

And these are only 5 curves... Í have about 15 curves to compare.

Sidenote

The obvious way to improve the plot is to show only a selection of the curves. I'm trying to focus on the most interesting ones in the paper, and banish the rest of them to the supplementary information section. The better I can make my plot, the least curves I would have to banish.

$\endgroup$
  • 1
    $\begingroup$ Related question, Color and line thickness recommendations for line plots. The plot would be much easier to digest if you just plotted the lines and did not include the error bars. A convenient way to include error bars is to make a set of small multiple plots, so the lines are not overlapping each other. $\endgroup$ – Andy W Feb 19 '15 at 13:41
  • $\begingroup$ Thanks for the link. Yes, removing the error bars would improve the plot, but my reviewers will immediately jump on me for not giving a reliability metric don't you think? If I make a set of multiple small plots, wouldn't that decease our ability to compare the lines? $\endgroup$ – Rodin Feb 19 '15 at 13:55
  • 1
    $\begingroup$ Sure it is harder to compare across panels than within panels, but that is a better alternative than a mess of superimposed lines in one panel. See my answer at the other linked question as well, you can often cluster similar trajectories in the same panel, which makes comparisons easier. Here you might put A, C and E in one panel (decreasing) and B and D in another (increasing). $\endgroup$ – Andy W Feb 19 '15 at 14:00
  • $\begingroup$ That's a good idea! The answers (including yours) on that other thread are treasure trove of information. Guess that makes this question a duplicate :) $\endgroup$ – Rodin Feb 19 '15 at 14:01
4
$\begingroup$

As was the recommendation in Color and line thickness recommendations for line plots, small multiples are a common solution for plots that have problems with overplotting. Here is an example with the 5 curves you provided.

enter image description here

It is a lot of information, but it is pretty easy to see that A, C & E are all decreasing. C & E have increasing variance for high parameter values, while B and D are fairly constant, and A hits the bottom (I would guess the metric has to be a positive value given the graphs.) Small multiples can extend to 15 curves, but when making the panels smaller IMO you should make the panels a bit more minimalist in the smaller space, e.g. ditch the gridlines, make the tick marks smaller and more sparse, etc.

Error bars make the overplotting problem even more problematic, so it is harder to stuff multiple error bars into one graph if the trajectories overlap. One alternative way I like though is to use semi-transparent areas as opposed to the points and bars. My labeling could use some work, but here is an example with these curves (plus and minus two standard deviations to make the areas a bit wider).

enter image description here

I had to put E in a separate panel, as it occluded A and C. Seeing the overlap of two areas is not that difficult given the right colors for the areas, three though is very difficult.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.