# How to present data in a large dataset (x is categorical, y is quantitative)?

I'm working on a computational biology project, and my professor has given me data for over 1,000 enzymes (represented by a numerical index), namely average solubility, count (number of enzymes of each type), and standard deviation.

I'm having some trouble determining how to best graphically summarize this data. My professor sent me a histogram/barplot(?) that ranked the enzyme indexes by average solubility.

So I guess I have two main questions at the moment:

1. What is the best way to present the average solubility for such a large number of categorical indexes? My thought is that a dot plot would be a better choice because it would use less "ink".

2. My professor wants to include information regarding the standard deviation. However, because there are so many data points, the inclusion of error bars just results in a black mass on the graph (whether it is a dot plot or a barplot). My professor made a scatterplot of average solubility against the standard deviation, but I'm skeptical that such a graph is very meaningful.

Edit: Just for better visual of the situation, here are the initial barplot and dot chart, both depicting average solubility by enzyme index:

• Hmm, the index does seem to mean something, doesn't it? It's no coincidence that the graph looks so smooth. – g3o2 Jun 5 '17 at 19:51
• The smoothness of the graph is because the indices were intentionally ranked by average solubility - or are you noticing something unusual? – statsStudent Jun 5 '17 at 20:29
• Ok, just wanted to make sure :-) – g3o2 Jun 5 '17 at 21:28

Your professor's suggestion sounds reasonable to me. You could include histograms of solubility and standard deviation too in a facet grid.

I like polar plots at the moment, so if you want something a bit fancier, how about a circular dot plot coloured and sized by standard deviation:

df <- data.frame(Enzyme = 1:1000, Solubility = rgamma(1000,10,1), sd =
rgamma(1000,2,5))

ggplot(data = df, aes(x = Enzyme, y = Solubility)) +
geom_point(alpha = 0.5, aes(colour = sd, size = sd)) +
coord_polar() +
theme_bw() +
scale_size_continuous(range = c(0.5, 3), guide = F) +
scale_color_gradient(low = "blue", high = "red", name = "SD")


which gets you

If my polar plot obsession is too much, just remove coord_polar() to get

• Those plots are gorgeous! My only concern, though, would be how well a colored dot plot of that variety would render in a printed research publication (likely in only black and white). – statsStudent Jun 5 '17 at 17:39
• They look OK with scale_color_gradient(low = "blue", high = "red", name = "SD") changed to scale_color_gradient(low = "black", high = "gray", name = "SD") too, I think. Another thought: you could try colouring the bar plot by SD and order the indices by SD to get a smooth SD gradient. – Will Jun 5 '17 at 19:18
• I'll have to try that out! Also, I wasn't familiar with facet grids in ggplot2 before - just realized you and g3o2 were thinking along the same lines with a small multiples presentation. Thanks! – statsStudent Jun 5 '17 at 20:35

If you have a large dataset along with a large number of variables, You might consider using correlation as well as factor analysis to ascertain relationships between variable and groups of variables. Not being a biologist, then kitchen sink approach of eigenvectors might glean some insight hidden insights.

• I actually have yet to use eigenvectors as applied to data science (have only seen them in an introductory linear algebra class so far!), but I'll definitely look in to that. – statsStudent Jun 5 '17 at 17:40

Ad 1) Ordering the enzymes by average solubility in descending order is a reasonable suggestion, unless the enzyme index has a meaning itself.

Ad 2) Try ordering the enzymes by descending coefficient of variation but show the average solubility on the y axis.

Alternatively, order the enzymes as in 1) but showing either standard deviation or CV. More generally, organize the plots as small multiples, the common axis being the enzymes ordered as in 1) but showing each time another variable on the y axis.

As for encoding marks, use nothing too complicated. Just stick with either dots, thin bars, dashes, etc. No need for error bars unless you want to display sampling error - even then, simple does it, the small multiple approach being cleaner and thus more effective.

If the enzyme index has a meaning, e.g. it would stand for a hierarchical classification number, then you should organize the enzymes by such groups first. If the index is just a hash, then the above suggestions apply.

Finally, you will face the challenge populating the x axis with useful labels. The easiest solution is to make your plot interactive, allowing a zooming axis and tooltips. Absent interactivity, you probably need to compromise on label detail.

• Small multiples seems like a simple and elegant way to present this - I hadn't thought of that. In Ad 2), why do you suggest using coefficient of variation rather than standard deviation? – statsStudent Jun 5 '17 at 17:43
• For 2), it depends on the domain of expertise. Feel free to choose whichever you like. For surveys, CVs are used to provide an idea of precision compared to the mean, while standard deviations are used to calculate confidence intervals. It really depends on your use case. – g3o2 Jun 5 '17 at 19:48