Are there any simpler alternatives to boxplots and barplots that can faithfully represent data with outliers and strongly varying ns?

Question

I have a dataset of passwords that looks like this:

My goal is to make a graph that accurately showcases the strength of each category so my initial plan was to group the data by category and calculate the average strength like this:

then represent the data with a barplot like this one:

My problem with this is that when I looked up the cardinality of each category I noticed that the number of data points is very different as seen here below:

So I thought that calculating the average blindly and making a barplot isn't very faithful to the data because of two reasons. First, you can't see the number of data points in a bar plot; secondly, you can't see the outliers that would immensely affect the mean, especially when you have a very small number of data points in certain categories. So I decided to try and make a boxplot instead as seen here:

But my problem with this is that a boxplot is more complicated for the average person than a simple barplot, even if it's more faithful to the dataset.

What do you guys think is better? A barplot or a boxplot? Is there any alternative that is simple and faithful to the dataset I described?

EDIT: First of all here is the variables concerned in the dataset for you guys to see the data :

> dput(passwords[c(3,8)])
structure(list(category = c("password-related", "simple-alphanumeric", 
"simple-alphanumeric", "simple-alphanumeric", "simple-alphanumeric", 
"simple-alphanumeric", "animal", "sport", "sport", "password-related", 
"animal", "simple-alphanumeric", "simple-alphanumeric", "cool-macho", 
"name", "cool-macho", "cool-macho", "name", "simple-alphanumeric", 
"simple-alphanumeric", "name", "name", "cool-macho", "simple-alphanumeric", 
"cool-macho", "simple-alphanumeric", "cool-macho", "cool-macho", 
"name", "name", "name", "sport", "name", "password-related", 
"password-related", "cool-macho", "sport", "name", "name", "name", 
"name", "fluffy", "fluffy", "name", "simple-alphanumeric", "food", 
"name", "password-related", "simple-alphanumeric", "simple-alphanumeric", 
"name", "name", "nerdy-pop", "fluffy", "name", "sport", "sport", 
"simple-alphanumeric", "name", "simple-alphanumeric", "simple-alphanumeric", 
"name", "fluffy", "rebellious-rude", "rebellious-rude", "nerdy-pop", 
"cool-macho", "cool-macho", "name", "fluffy", "name", "cool-macho", 
"fluffy", "cool-macho", "name", "name", "simple-alphanumeric", 
"nerdy-pop", "name", "simple-alphanumeric", "sport", "food", 
"fluffy", "name", "name", "name", "name", "fluffy", "fluffy", 
"animal", "rebellious-rude", "simple-alphanumeric", "cool-macho", 
"cool-macho", "cool-macho", "name", "nerdy-pop", "animal", "fluffy", 
"name", "cool-macho", "name", "fluffy", "cool-macho", "animal", 
"simple-alphanumeric", "name", "animal", "cool-macho", "name", 
"cool-macho", "fluffy", "sport", "fluffy", "name", "simple-alphanumeric", 
"cool-macho", "name", "fluffy", "sport", "name", "nerdy-pop", 
"nerdy-pop", "rebellious-rude", "cool-macho", "name", "password-related", 
"name", "sport", "sport", "animal", "cool-macho", "simple-alphanumeric", 
"password-related", "name", "name", "cool-macho", "cool-macho", 
"name", "nerdy-pop", "food", "fluffy", "name", "animal", "simple-alphanumeric", 
"password-related", "name", "cool-macho", "nerdy-pop", "nerdy-pop", 
"name", "name", "cool-macho", "password-related", "name", "name", 
"sport", "food", "cool-macho", "animal", "sport", "cool-macho", 
"animal", "password-related", "name", "name", "name", "cool-macho", 
"name", "sport", "nerdy-pop", "name", "name", "animal", "cool-macho", 
"name", "sport", "nerdy-pop", "cool-macho", "name", "name", "sport", 
"rebellious-rude", "cool-macho", "name", "name", "name", "cool-macho", 
"fluffy", "sport", "name", "nerdy-pop", "simple-alphanumeric", 
"name", "rebellious-rude", "sport", "nerdy-pop", "simple-alphanumeric", 
"name", "animal", "simple-alphanumeric", "sport", "name", "simple-alphanumeric", 
"name", "name", "simple-alphanumeric", "name", "fluffy", "name", 
"cool-macho", "name", "fluffy", "rebellious-rude", "sport", "name", 
"fluffy", "name", "name", "fluffy", "name", "name", "fluffy", 
"name", "name", "name", "name", "nerdy-pop", "cool-macho", "cool-macho", 
"name", "food", "rebellious-rude", "name", "sport", "name", "cool-macho", 
"name", "nerdy-pop", "name", "name", "fluffy", "fluffy", "fluffy", 
"name", "food", "animal", "name", "name", "sport", "name", "name", 
"name", "name", "name", "sport", "sport", "name", "name", "name", 
"name", "simple-alphanumeric", "name", "fluffy", "cool-macho", 
"sport", "name", "name", "name", "password-related", "password-related", 
"fluffy", "name", "nerdy-pop", "cool-macho", "animal", "simple-alphanumeric", 
"simple-alphanumeric", "animal", "name", "name", "fluffy", "food", 
"fluffy", "fluffy", "simple-alphanumeric", "simple-alphanumeric", 
"name", "name", "name", "name", "simple-alphanumeric", "name", 
"nerdy-pop", "cool-macho", "name", "cool-macho", "cool-macho", 
"rebellious-rude", "animal", "cool-macho", "name", "food", "animal", 
"simple-alphanumeric", "fluffy", "name", "animal", "fluffy", 
"sport", "simple-alphanumeric", "name", "name", "sport", "simple-alphanumeric", 
"simple-alphanumeric", "food", "name", "animal", "password-related", 
"nerdy-pop", "cool-macho", "name", "cool-macho", "simple-alphanumeric", 
"name", "name", "cool-macho", "cool-macho", "password-related", 
"name", "name", "nerdy-pop", "name", "name", "nerdy-pop", "cool-macho", 
"food", "fluffy", "name", "name", "name", "name", "simple-alphanumeric", 
"cool-macho", "name", "name", "animal", "fluffy", "cool-macho", 
"cool-macho", "simple-alphanumeric", "simple-alphanumeric", "name", 
"name", "animal", "animal", "cool-macho", "nerdy-pop", "sport", 
"sport", "name", "sport", "animal", "simple-alphanumeric", "name", 
"nerdy-pop", "simple-alphanumeric", "fluffy", "cool-macho", "animal", 
"simple-alphanumeric", "cool-macho", "simple-alphanumeric", "name", 
"fluffy", "fluffy", "cool-macho", "name", "name", "cool-macho", 
"cool-macho", "name", "cool-macho", "name", "name", "cool-macho", 
"name", "cool-macho", "name", "sport", "name", "name", "name", 
"simple-alphanumeric", "cool-macho", "name", "name", "name", 
"fluffy", "name", "name", "simple-alphanumeric", "fluffy", "fluffy", 
"sport", "simple-alphanumeric", "name", "name", "simple-alphanumeric", 
"name", "name", "cool-macho", "simple-alphanumeric", "cool-macho", 
"fluffy", "sport", "rebellious-rude", "cool-macho", "name", "simple-alphanumeric", 
"nerdy-pop", "cool-macho", "animal", "name", "cool-macho", "cool-macho", 
"simple-alphanumeric", "name", "simple-alphanumeric", "cool-macho", 
"cool-macho", "cool-macho", "name", "name", "name", "animal", 
"fluffy", "cool-macho", "name", "nerdy-pop", "name", "name", 
"sport", "name", "nerdy-pop", "sport", "name", "name", "sport", 
"fluffy", "name", "cool-macho", "simple-alphanumeric", "cool-macho", 
"animal", "nerdy-pop", "nerdy-pop", "simple-alphanumeric", "name", 
"name", "name", "simple-alphanumeric", "cool-macho", "animal", 
"fluffy", "animal", "name", "name", "name", "name", "name", "simple-alphanumeric", 
"cool-macho", "name", "cool-macho", "name", "simple-alphanumeric", 
"fluffy", "rebellious-rude", "name", "nerdy-pop", "name", "name", 
"name", "sport", "name", "food", "name", "simple-alphanumeric", 
"name", "cool-macho", "name", "nerdy-pop", "cool-macho", "cool-macho", 
"name", "nerdy-pop", "name", "password-related"), strength = c(8, 
4, 4, 4, 8, 4, 8, 4, 7, 8, 8, 1, 32, 9, 9, 8, 8, 9, 0, 0, 8, 
10, 8, 4, 8, 25, 7, 8, 8, 6, 7, 6, 7, 4, 3, 6, 8, 6, 9, 8, 8, 
6, 9, 7, 4, 0, 8, 3, 4, 4, 8, 6, 8, 8, 7, 5, 7, 7, 8, 0, 4, 6, 
8, 7, 7, 10, 6, 9, 8, 7, 8, 6, 6, 8, 8, 8, 0, 8, 7, 1, 8, 5, 
8, 8, 8, 9, 8, 8, 6, 7, 7, 5, 8, 7, 7, 8, 8, 8, 9, 8, 8, 9, 7, 
7, 9, 0, 8, 8, 9, 8, 10, 6, 7, 8, 8, 1, 7, 8, 9, 8, 8, 6, 6, 
9, 8, 8, 8, 7, 8, 7, 8, 7, 8, 7, 7, 6, 8, 9, 7, 8, 3, 0, 6, 7, 
0, 8, 7, 8, 46, 8, 6, 9, 8, 8, 6, 8, 6, 0, 9, 8, 8, 6, 6, 6, 
8, 8, 7, 7, 6, 5, 8, 7, 7, 7, 8, 8, 8, 8, 8, 8, 7, 8, 7, 7, 7, 
8, 6, 9, 6, 8, 0, 8, 0, 8, 2, 8, 46, 6, 8, 9, 9, 8, 8, 8, 8, 
9, 0, 8, 7, 6, 6, 9, 8, 8, 7, 6, 8, 6, 8, 8, 8, 9, 7, 9, 8, 8, 
5, 8, 8, 6, 9, 6, 6, 7, 6, 8, 8, 7, 8, 6, 8, 8, 7, 4, 8, 6, 7, 
8, 8, 9, 6, 7, 6, 8, 5, 8, 9, 9, 7, 8, 1, 0, 7, 8, 9, 8, 6, 7, 
8, 7, 6, 7, 7, 19, 6, 8, 8, 0, 7, 8, 6, 4, 7, 7, 7, 0, 0, 5, 
6, 8, 9, 0, 10, 9, 8, 7, 9, 2, 5, 7, 7, 8, 6, 6, 10, 7, 4, 10, 
9, 6, 4, 6, 8, 8, 1, 4, 5, 8, 9, 8, 38, 6, 8, 0, 0, 7, 6, 5, 
6, 6, 8, 8, 9, 5, 6, 48, 3, 4, 8, 9, 6, 7, 4, 35, 7, 7, 8, 7, 
6, 6, 8, 0, 0, 5, 6, 9, 6, 7, 36, 6, 8, 3, 5, 7, 0, 7, 8, 4, 
7, 7, 7, 0, 7, 0, 7, 4, 9, 8, 7, 8, 10, 9, 6, 6, 7, 4, 3, 7, 
10, 6, 7, 6, 6, 7, 36, 6, 4, 10, 5, 5, 7, 8, 0, 6, 7, 34, 21, 
8, 8, 0, 5, 7, 3, 4, 7, 7, 8, 8, 7, 5, 0, 8, 9, 9, 8, 8, 6, 7, 
7, 4, 8, 8, 8, 9, 8, 9, 6, 4, 9, 6, 7, 8, 7, 8, 8, 7, 8, 8, 4, 
10, 8, 9, 7, 4, 8, 8, 7, 8, 0, 6, 8, 7, 32, 6, 7, 4, 6, 8, 4, 
7, 7, 9, 0, 8, 8, 8, 8, 36, 6, 8, 8, 8, 8, 6, 8, 7, 8, 6, 6, 
0, 7, 8, 7, 7, 7, 6, 9, 7, 7, 28)), row.names = c(NA, -500L), class = c("tbl_df", 
"tbl", "data.frame"), na.action = structure(501:507, .Names = c("501", 
"502", "503", "504", "505", "506", "507"), class = "omit")

here is the boxplot I made after taking some of your suggestions. What do you think I should change?

Answer 1

Concrete suggestions:

1. Alphabetical order is not helpful. Order categories according to their medians.

2. Use log scale. More than half your graph space is devoted to showing the 12 highest values out of 500 observations, which is not using space efficiently. Some of your distributions are left-skewed but they won't suffer much under log scale. (Note: If there are exact zeros in your data, something like log (value + 1) should be fine for visualization.)

3. Show all the data as points: even if they blur together, you will see some fine structure. For example, it looks as if more than 25% of the simple-alphanumeric passwords all tie with the lowest possible strength (zero?).

You hadn't posted your data at the time of first writing, but here is a loosely comparable dataset with 11 categories and 500 observations.

The display is a quantile-box plot in the sense of Emanuel Parzen. As @Gung suggests in his answer, although perhaps for different reasons, I start with just the boxes, medians and quartiles. To a good enough approximation, calculating medians and quartiles commutes with taking logarithms, so that e.g. log(median) will almost always be close to, if not identical with, median(log) (*). But then I do not bother with the usual fal-lal about drawing whiskers to the furthest points within 1.5 IQR of the nearer quartile and plotting as individual points any values that lie beyond. That is because I plot all the data and am happy that moderate or severe outliers will separate themselves from any crowd. That is how they are defined and recognised.

In my case the data arrive on a natural logarithm scale. In your case you need to arrange that your favourite software shows vertical axis labels that make sense, such as 0 (if needed), 1, 2, 5, 10, 20 and 50.

(*) The small print arises whenever medians and quartiles are calculated by interpolation between data values.

EDIT Your data are quite a challenge to an occasional R user like myself. (Yes, I know I could install it and have done so many times.) Here is the graph I proposed for your data, produced using Stata. Putting the names vertically is against my best instincts, but swapping axes does not help much here. The main plus point about this graph -- other than underlining that the medians are mostly about the same -- is to show some of the fine structure. The scale is log(strength + 1) with (tick) axis labels for strength.

EDIT 2

Here is the left-hand category (simple-alphanumeric) expanded to show how the plot works.

Individual values are sorted in order. So, we can see repeated values as such; and the horizontal scale is a rank or more precisely a sort from smallest to largest. The median and quartiles are defined as usual. The box bears this interpretation (with small print for ties): half the values are inside the box and half outside. The left-hand display uses log(strength + 1) and the right-hand display uses a plain strength scale.

EDIT 3

Yet another attempt follows. Here I am mindful that

• Logarithmic scale (especially the variant based on adding 1 first) often does not appeal or is not well understood.

• There is need to show fine structure, not just the outliers but also detail such as the many zeros for some categories. Much of that is implicit in the box plots, but questions here on CV show that box plots for discrete variables with many repeated values are often found puzzling or even misinterpreted.

• There is need to convey the different sample sizes, which I think this version does as well or better than any previously posted.

The one simplification made here is to lump several groups with similar distributions together.

I call this a quantile plot, but that is a technical term and I would explain it to lay audiences as plotting values in order. Labelling in terms of percentile rank is based on an idea that plotting position or cumulative probability are less likely to be familiar terms. (Those wanting sympathy for non-statistical audiences might not know that I am not a statistician, but a geographer.)

Answer 2

I agree with you that a box plot alone doesn't describe the data very well. But a box plot overlaid with the data points might do. See for example the graph below. I've switched the axis around so it's easier to read, jittered the points and added tick marks so the exact values can be read, added the overall median for comparison and indicated the N on the category axis. The transformation is square root, not as extreme as log and allows you to see the low and the high end data well. The categories are sorted by median score.

R Code follows:

pass$medianstrength <- ave(pass$strength , pass$category, FUN=median)
pass$counts <- ave(pass$strength , pass$category, FUN=length)

ggplot(pass, aes(y=reorder(sprintf("%s (N=%d)", category, counts), -medianstrength), x=strength, col=category)) + 
 geom_boxplot(outlier.color = NA,col="black")  + 
 geom_point(position = position_jitter(height = 0.3,width=0.01),alpha=0.5) + 
 scale_x_sqrt(breaks=c(0,1,2,3,4,5,6,7,8,9,10,20,30,40,50)) + 
 theme_bw() + 
 theme(legend.position = "none") + 
 geom_vline(xintercept=median(pass$strength), lty="dashed") + 
 labs(x="Strength", y="Category") 

Edit following the suggestion from @bjorn to use geom_sina:

I've also removed the whiskers from the box, filled the box a light grey and added the median more prominently for each group. The height of the box reflects the sample size in the category.

A potential problem here is the horizontal jitter, I should have made it proportional to the square root of the value to avoid the very large jitters at the low end, and applied it to the points only not the data as a whole.

Code:

pass$medianstrength <- ave(pass$strength , pass$category, FUN=median)
pass$counts <- ave(pass$strength , pass$category, FUN=length)

ggplot(pass, aes(y=reorder(sprintf("%s (N=%d)", category, counts), -medianstrength), x=strength+runif(seq_along(strength),-0.2,0.2), col=category)) + 
  geom_boxplot(outlier.color = NA,col=NA,coef=1,fill=hsv(0,0,0,.2), varwidth=TRUE)  + 
  ggforce::geom_sina(alpha=0.6) + 
  geom_errorbar(aes(xmin=medianstrength, xmax=medianstrength), col="black",alpha=0.5, width=0.5) + 
  scale_x_sqrt(breaks=c(0,1,2,3,4,5,6,7,8,9,10,20,30,40,50)) + 
  theme_bw() + 
  theme(legend.position = "none") + geom_vline(xintercept=median(pass$strength), lty="dashed") + 
  labs(x="Strength", y="Category") 

Answer 3

I gather there are several issues here: 1) there is a wide range of amounts of data for the categories; 2) you suspect outliers are distorting the category means; 3) you suspect boxplots are too sophisticated for your audience.

In general, I think boxplots are superior to barplots and that they convey more information. But if you think your audience will be overwhelmed by them, they're out. If you think a good figure caption would be enough to make them intelligible, I would try for that. The outliers expand the range of the plot so much as to make the bulk of the distributions hard to see, though, so I might try just plotting the 'boxes', i.e., the medians and quartiles. That would let you zoom in on the data.

If you think boxplots are not salvageable, you will need to fall back to barplots. Barplots don't have to display means, though. You could make them for the medians, or you could make them for trimmed means. A $20\%$ trimmed mean is the mean of the middle $60\%$ of your data after trimming off the top and bottom $20\%$. You would need a figure caption to explain that and, of course, you could use some other trimming percentage as you find appropriate. Again, if you think it's too complicated, just use medians (a.k.a., $50\%$ trimmed means). Either approach will minimize the distortion caused by the outliers.

Varying the width of boxplots or bars is a common way to represent different amounts of constituent data, as @whuber suggests. It is common for software to allow this. A common recommendation is to make the widths proportional to the square root of the $n$s. Once again, this will require a figure caption to explain it, so there's always still a tradeoff between the ability of the plot to represent information and the required sophistication on the part of the viewers.

Answer 4

Here is an attempt at visualizing the password strength data by following the "less is more" principle.

The idea is simple:

• Create five strength categories: "very weak [0-4]", "weak [5-6]", "medium [7-8]", "strong [9-10]", "very strong [11+]". I came up with these groups by looking at the histogram of strength values but you might have a better idea for how to split the numeric range of strength into bins. The "very strong" passwords are the "outliers".
• Make a bar plot for each password category.
• Order the bar plots by increasing average strength.

For comparison (and since otherwise there is a gap in the 3x4 grid of panels by password category) I add the overall bar plot as well.

This visual has a lot less detail than those proposed by @NickCox and @GeorgeSavva but it's easier (at least for me) to compare the password categories qualitatively. For example:

• "food" is the worst choice as it never results in a strong password.
• "simple-alphanumeric" is the second worst choice because it's risky: it can generate a (very) strong password but many choices are very weak. NB: The alphanumeric password "abc123" has strength 32 but I have doubts that it's a good password choice.
• "rebellious-rude", "name", "cool-macho" and "sport" are mostly medium strength.
• "nerdy-pop" is the safest choice.