# How can I remove the z-order bias of a coloured scatter plot?

I'm currently working on a plot engine for my project. This engine should be robust for a wide range of inputs. In order to analyse the data, I'm plotting a series of graphs utilising python/matplotlib. Among them is the following:

I think this graph is not good because the data being plotted first (high pressures, red) have a lower z-order (i.e. they are overdrawn) than the blue bullets for low pressures. Thus introducing a bias when looking at the graph. The underlying reason for that is that the data is bell-shaped.

First off, do you agree or disagree? I could leave it like it is because it is just one of many views on the data. It could still be useful.

However, if there is a way to make this graph better with some sort of trick, I'd be much happier. I already played with point size, transparency/alpha and edgecolor. This only made it worse. A great way to remove the z-order bias in scatter plots is to bin the data and colour-code it accordingly (e.g. hexbin). But since I used the colour for the pressure information, I see no possibility to to something similar.

Another idea would be to randomise the z-order, but I'm not sure how to do that and if the result would be better.

Any other comments for improvements are appreciated.

• One can also look at the code if interested. – Sebastian Jun 21 '11 at 6:28
• Not the question, but the rainbow colour scheme here, although frustratingly popular, is a flawed choice: it isn't monotonic in terms of perception and it poses problems for many people who have difficulties with colours, especially red and green. agupubs.onlinelibrary.wiley.com/doi/abs/10.1029/2004EO400002 is one reference. – Nick Cox Mar 29 at 10:26
• Most people are interpreting this as a question about Python code. If so, it is off-topic here according to current guidelines. – Nick Cox Mar 29 at 10:28

First off, I agree.

I suspect that you can create a different sort of graph; you're not using a lot of the two-dimensionality of the current display because everything is clustered about the x=y line. Try plotting the pressure along the x axis and the ratios along the y axis. If this is too messy, try taking the difference in pressure. You could also use some measure of effect size, like Cohen's d, but then viewers would have to know what that is. You can probably come up with something better than what I suggested, but my suggestion might help you think of other approaches. As you'll read below, my approach might mislead viewers because it would make pressure look like an independent variable.

It would help to know what sort of story you're telling from this graph. My interpretation is that the ratios are independent variables and the pressure is a dependent variable. The change that I suggested above makes it look like the pressure is independent and the ratios are dependent. (That might not be a problem.)

But here are a ideas that use your current graph.

• Sorting a list randomly in python
• It looks like the pressures might be clustered a bit. I'm not sure whether this is what you were saying was bell-shaped. But if they are clustered, you could try assigning different dot types to each of a small number of clusters
• For each of the axes, plot a histogram of that variable with the pressure colors stacked on top of each other. Even if you don't change the main three-variable plot, these two-variable modified histograms would help point out the bias in the display.
• Plot difference versus mean whenever equality of measures is a reference state. Then difference = 0 corresponds to y = x and the space is used more effectively. – Nick Cox Mar 29 at 10:30

Unfortunately there isn't really a perfect solution to this, mainly because z-order cannot be decoupled draw order (see this question).

As a work-around you can have a wrapper class, which allows to make lazy draw calls, and internally randomizes the draw call order. For example:

class ZBiasFreePlotter(object):
def __init__(self):
self.plot_calls = []

def add_plot(self, f, xs, ys, *args, **kwargs):
self.plot_calls.append((f, xs, ys, args, kwargs))

def draw_plots(self, chunk_size=512):
scheduled_calls = []
for f, xs, ys, args, kwargs in self.plot_calls:
assert(len(xs) == len(ys))
index = np.arange(len(xs))
np.random.shuffle(index)
index_blocks = [index[i:i+chunk_size] for i in np.arange(len(index))[::chunk_size]]
for i, index_block in enumerate(index_blocks):
# Only attach a label for one of the chunks
if i != 0 and kwargs.get("label") is not None:
kwargs = kwargs.copy()
kwargs["label"] = None
scheduled_calls.append((f, xs[index_block], ys[index_block], args, kwargs))

np.random.shuffle(scheduled_calls)

for f, xs, ys, args, kwargs in scheduled_calls:
f(xs, ys, *args, **kwargs)


The usage would be:

fig, ax = plt.subplots(1, 1)
bias_free_plotter = ZBiasFreePlotter()

for xs, ys, color in things_to_plot:

bias_free_plotter.draw_plots()


For example, this is my plot with standard z-order bias:

And this is what I get using lazily cached plot calls and applying some alpha overdraw:

A few caveats:

• Only works for marker plots, line plots will get messed up from the chunking.
• Every plot advances the color cycle, i.e., with this approach it is necessary to control the color cycle manually, which can be done by something like this.

Did you plot each group of data using a separate call to scatter?

If you plot all your data in one pass, I think the overlaying of dots on other dots would only come from the order in which there were plotted (not 100% sure though, can't test at the moment) So, one way to get different layouts would be to randomly permute your data before plotting it, or even sort them in a non-random way, to get the desired effect.

Could you paste your source code?

• thanks for your answer. I used only one scatter call. I agree that the the order is determined by what is plotted first. Here's the code – Sebastian Jun 21 '11 at 6:27