Q: Does there exist experimental evidence supporting Tufte-style, minimalist, data-speak visualizations over the chart-junked visualizations of, say, Nigel Holmes?

I asked how to add chart-junk to R plots here and responders threw a hefty amount of snark back at me. So, surely, there must be some experimental evidence, to which I'm not privy, supporting their anti-chart junk position---more evidence than just "Tufte said so." Right?

If such evidence exists it would contradict a lot of psychological research we have regarding humans, their memory recall, and pattern identification. So I'd certainly be excited to read about it.

A little anecdote: at a conference I asked Edward Tufte how he regards experimental evidence finding that junk animations and videos improve humans' understanding and memory recall [see research cited in Brain Rules]. His response: "Don't believe them." So much for the scientific method!

P.S. Of course, I'm needling people a little here. I own all of Tufte's books and think his work is incredible. I just think that his supporters have oversold some of his arguments.

NOTE: This is a re-post of a question I asked on StackOverflow. Moderators closed it because it wasn't programming-specific. CrossValidated might be a better home.

UPDATE: There are some useful links in the comments section of my original question post---namely, to the work of Chambers, Cleveland, and the datavis group at Stanford.

UPDATE: This question deals with similar subject matter.

  • 7
    $\begingroup$ Do you mind citing the evidence that Tufte/minimalist style charts contradicts all psychological research we have regarding humans, their memory recall, and pattern recognition? While I think this is a good question, such a negative and condescending tone does not make your request appear very sincere. Nor does spending 10 minutes to do background research to refute my suggestion of reading Cleveland's work as pertinent to the discussion. $\endgroup$
    – Andy W
    Nov 15 '11 at 19:28
  • 2
    $\begingroup$ @AndyW I did link "Brain Rules" and a discussion of Nigel Holmes' research. Here is another link supporting my claim about fonts, back in the original comments section. I could go on, but the basic point is that the brain will engage with, understand, and remember better visuals that excite and challenge it. But this is based on my reading of PopPsych... $\endgroup$
    – lowndrul
    Nov 15 '11 at 19:50
  • $\begingroup$ @AndyW ...I can't pretend to be an authority on the subject. I'm hoping someone more knowledgeable will chime in on this. Also, my question/claim was admittedly bold. I wanted to elicit a response. I watered it down a little so that it does not read as negative. Also, something must have been lost in translation. I DID think your links to Cleveland's work were relevant---hence, my mention of it in the "UPDATE" to my question. $\endgroup$
    – lowndrul
    Nov 15 '11 at 20:01
  • $\begingroup$ I think the reason for the snarkiness was your use of Excel as a point of reference/starting-point. They weren't ragging the charts, they were ragging Excel. $\endgroup$
    – bill_080
    Nov 15 '11 at 20:36
  • $\begingroup$ In response to @bill_080, I'm sure some people were ragging the charts; I would have if I'd chimed in. But perhaps the snark was unjustified; I, for one, haven't read any of the evidence one way or the other. Good question! $\endgroup$ Nov 15 '11 at 21:25

The literature is vast. Experimental evidence is abundant but incomplete. For an introduction that focuses on the psychological and semiotic investigations, see Alan M. MacEachren, How Maps Work (1995; 2004 in paperback). Jump directly to chapter 9 (near the end) and then work backwards through any preliminary material that interests you. The bibliography is extensive (over 400 documents) but is getting a little long in the tooth. Although the title suggests a focus on cartography, most of the book is relevant to how humans create meaning out of and interpret graphical information.

Don't expect to get a definitive answer out of any amount of such research. Remember that Tufte, Cleveland, and others were primarily focused on creating graphics that enable (above all) accurate, insightful communication of and interpretation of data. Other graphics artists and researchers have other aims, such as influencing people, creating effective propaganda, simplifying complex datasets, and expressing their artistic sensibilities within a graphical medium. These are almost diametrically opposed to the first set of objectives, whence the hugely differing approaches and recommendations you will find.

Given this, I think a review of Cleveland's research should be sufficiently convincing that many of Tufte's design recommendations have decent experimental justification. These include his use of the Lie Factor, the Data-Ink Ratio, small multiples, and chartjunk for critically evaluating and designing statistical graphics.

  • 9
    $\begingroup$ (+1) Your 2nd paragraph reminds me of a recent discussion (with Gelman, Kosara, Wickham, among others) about infographics vs. data visualization, e.g. Information visualization” vs. “Statistical graphics, Infovis, infographics, and data visualization: Where I’m coming from, and where I’d like to go or Statistical Graphics and Information Visualization. $\endgroup$
    – chl
    Nov 15 '11 at 21:09
  • $\begingroup$ +1 Thanks especially for the second paragraph. As with so much of applied statistics, the answer depends on why the question was asked and who asked it. (Not that this diminishes the importance of evidence; thanks for the question, brianjd!) $\endgroup$ Nov 15 '11 at 21:20
  • $\begingroup$ (+1) to both for useful pointers. @chl your 1st and 3rd links are the same. Did you intend to link to this for the 3rd? $\endgroup$
    – lowndrul
    Nov 15 '11 at 21:29
  • $\begingroup$ Just my intuition chiming in here (I haven't read the reference), but I'd reckon that it wouldn't take a particularly in-depth study to show that Tufte-style boxplots (two bars, and a centre point) are far less understandable than a standard boxplot (which has it's own issues). The extra ink doesn't add more data, but it does add more visual mass, which makes it more readable. The data-ink ratio principle is good, and can be wielded valiantly in the face of ostentatious chart junk, but it's not absolute, and should take into consideration the limitations of the human visual perception system. $\endgroup$
    – naught101
    Nov 17 '12 at 1:05
  • $\begingroup$ To test your intuition, @naught101, I invite you to examine the example at stats.stackexchange.com/a/13915. Tufte's design principles work well there because many boxplots need to be displayed and compared: the extra ink in standard boxplots interferes with the comparison. $\endgroup$
    – whuber
    Nov 17 '12 at 2:50

Here's some;

  • Cleveland and McGill (1984, JASA) Graphical Perception: Theory, Experimentation, and Application to the Development of Graphical Methods
  • Cleveland and McGill (1987, JRSSA) Graphical Perception: The Visual Decoding of Quantitative Information on Graphical Displays of Data
  • Lewandowsky and Spence (1989) Discriminating Strata in Scatterplots
  • Spence and Lewandowsky (1991) Displaying Proportions and Percentages
  • Spence Kutlesa and Rose (1999) Using Color to Code Quantity in Spatial Displays

Ask the Google for the full references

  • 2
    $\begingroup$ from LewSpe91 abstract: "An analysis of the nature of the task and a review of the psychophysical literature suggest that the traditional prejudice against the pie chart is misguided." EXACTLY! Well, that result surprised me. But that's the point: need to apply the scientific method, rather than datavis dogma, in determining what are the "best" ways to visualize data. If we do I'm sure there will be more surprising results. $\endgroup$
    – lowndrul
    Nov 28 '11 at 17:54

It's worth remembering that information visualisation isn't some island cut off from all other forms of visual communication. If you want to produce work based on evidence based princples, I'd argue it's best to look where the evidence is strongest.

I've read specific research on data visualisation techniques, and general research in cognitive science and in general design research, and I find that thinking through how the more powerful, more thorough general research applies to each brief and each element used is often more effective and useful than trying to apply the narrowly applied field-specific research which often suffers from small samples, weak research techniques, narrow investigation and/or deeply ingrained assumptions.

There are two excellent books I recommend as an introduction, one with the science as a starting point, one with general principles as a starting point, bringing in evidence:

  • Vision Science by Steve Palmer. It's a beast, and as a student I nearly gave myself a back injury on the few occasions I was foolish enough to carry it in a backpack, but it's also possibly the best science textbook I've ever seen, and a great example of crisp visual and verbal communciation itself. I went through it recently to label out the chapters with content directly relevant to my work in visualisation and information design, expecting to only label a few: I ended up labelling every chapter except one.
  • Universal Principles of Design by Rockport Press. A very ambitious and useful book which crunches cognitive science research with case studies and examples from across all branches of design into a series of awesomely clear and straight to the point double-page spreads, each covering one established, evidence based and practical principle, with practical suggestions, worked examples and suggested further reading. Very stimulating, so long as you think of it as a list of tools with suggested uses not a list of rules.

The only downside is, this approach takes more thinking to see how such principles are applicable. If you're looking for a list of arbitrary rules, as many in the data vis community seem to be, I'd say there isn't one and never will be except where people make massive unjustified assumptions and generalisations, or make things up. The better quality applied research is useful, but it helps to have a solid framework which it can slot into.

Most of Tufte's general principles such as data-ink and chart-junk can be traced back to solid general principles such as signal-noise ratios, figure-ground, attenuation, and others - but on the route to becoming field-specific and prescriptive, they have been combined with hefty assumptions and generalisations about your objectives and audience that turn them into blunt tools. Many of the apparent contradictions and debates in the applied research aren't contradictions at all if you take a step back, take context into account and work through from the underlying core principles and the particular features of each case.


There was one really good study in the field of cartography (Hegarty et al. (2009): Naïve cartography: How intuitions about display configuration can hurt performance. Published in: Cartographica The International Journal for Geographic Information and Geovisualization 44(3):171-186)

It is especially interesting as the authors looked at a more complex task than simply reading values of a bar chart: Expert and novice users had to determine wind speeds and pressure gradients from a meterological map. Both groups of participants intuitively preferred a map with added relief shading and state borders (something Tufte would probably refer to as “chartjunk” as it is irrelevant to the task) against a more minimalistic map showing only the outline of America in the background. But even though there was such a strong personal preference for chartjunk, participants actually performed significantly worse using this map, both in terms of accuracy and response time.

What I found particularly interesting about this study is that a complex use case (like meterology) is really common for us designers/cartographers/data analyst. Often times it is not just about some little bar chart, but we need to design entire dashboards, thematic maps, Sankey diagrams,... Cutting down on your chart junk does have improve your visualizations in this context a lot of times.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.