7
$\begingroup$

Every week we have new customers sign up, and some of them submit support requests. I'd like to graph the change in the number of support tickets per customer over time. I'm doing this by bucketing customers into one-week cohorts, and then graphing each distribution as a Tufte box plot, as in this image. The graph I have now looks something like this:

tufte graph

The problem is that the distribution of real data, instead of the fake data I'm using here, resembles an exponential decay with only integer values, and is thus skewed toward "zero tickets". So we have 52 cohort distributions that look like this:

exponential decay

(where the X axis is the number of tickets, and the Y axis is the number of customers who sent in that many tickets), making it tricky to get the actual distribution from the box plot. In general, the median doesn't tell you a whole lot about the distribution.

How could I better show the series of exponential decay distributions, to get a grasp of how the shape is changing over time?

Thanks, Kevin

$\endgroup$
6
$\begingroup$

One simple adjustment to your current graphic production would be, instead of producing all of the elements of the box-plot (even the minimalist Tufte style one), would be to produce a line chart connecting the summary statistics that the box plot is displaying (median, quartiles, mean, outer hinges, whatever). Below is an example displaying the 90th and 99th percentile of a simulated distribution of 50 observations over 100 weeks.

enter image description here

This connecting of the lines allows you to make the temporal connections between the summary statistics between weeks much easier, and reduces the data to ink ratio of the plot. Even Tufte in the Visual Display of Quantitative Information has an example where connecting the lines in a display allows one to discern periodicity in a temporal series that would be very difficult to see in a scatterplot display (and I assume the same problem would extend to the box-plot display).

What exactly you should display in the lines I believe would take more insight in the nature of your data and what you are interested in (and could change as the nature of your data changes over time as well). To get a broad sense of the distribution I believe different quantiles (including the median) can be informative. Although it may take some experimentation to see where informative quantiles lie from week to week (in this display the 99th percentile is quite noisy).

Line charts like this could also be extended to different statistical summaries (such as the skew of the distribution), although I believe the quantiles as a first run are the most informative. Also if you are interested in identifying outliers you may want to consider including the dots of outliers (defined in whatever way suits your fancy) in these same line plots. There was an interesting discussion of outliers for skewed data in this question on the site, Is there a boxplot variant for Poisson distributed data?, and the questions tagged with I believe would be applicable.

Even though from just this you can probably imagine generating a multitude of different lines on one plot, it is fairly easy to plot too much information in one graphic. A rule-of-thumb I try to abide by is that a plot should not have any more than 4-5 data elements (where here a data element would be a line). Even that is frequently too many. To attempt to get around this problem, try to make a consistent template, in which the axis of the plots are consistent, so you can make accurate comparisons between plots. Or if your software allows it, make a series of small multiple plots (again using the same axis for all plots). Then even if you say have the outer-hinges and outliers on one plot and the median and quartiles on another you may be able to discern patterns between those two plots. And then you can combine the outliers and the quartiles into one plot for more scrutiny if you think you see a pattern between them.

EDIT: As an example of the smoothing that @whuber is talking about here is a similar plot to that above (generated by a simulated process in the exact same manner), except that a loess smoother is applied to the lines.

enter image description here

I couldn't bring myself to not plot the original data, but I just made the smoothed lines thicker (and gave them color) to bring them to the foreground of the image, and left the original lines thinner and a light grey color so they are merely in the background of the image (and hence are not as distracting). The smoother allows one to assess general trends that can be obfuscated by the variance of the series.

Tukey has other suggestions I did not display here, such as plotting the alpha hull of all the observations (and labeling those observations that make up the vertices). Some more food for thought.

$\endgroup$
  • 1
    $\begingroup$ Tukey discusses this approach--and elaborates on it extensively--in his EDA text. He calls the curves "traces" and the whole thing is a "wandering schematic plot." After constructing the raw traces, he applies a (robust, nonparametric) smoother to them. See Chapter 8. $\endgroup$ – whuber Nov 23 '11 at 17:19
  • $\begingroup$ +1 thorough, clear, & illustrated. As an additional note, you seem a little self-conscious about plotting the original data in the background. I've always believed this the way to go--thinner and gray with the trends on top makes the graphic much more useful. $\endgroup$ – gung - Reinstate Monica Nov 29 '11 at 4:16
1
$\begingroup$

Have you considered perhaps focusing on having a 2D plot of number of tickets on the x-axis vs number of people that have submitted that many tickets on the y-axis, and showing each cohort on a plot by itself? Then you could flip from cohort to cohort (like a Rolodex) to get a sense of change over time using animation, or let time be the z-axis in a 3D plot.

(edit: fixed y-axis from "number in cohort" to "number who have submitted that many tickets")

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.