# Visualization strategy for sparse binary data

I have ≈90 000 medical cases with ≈250 distinct diagnoses between them, distributed over 20 years; the most common of which applies to roughly half, and the least common applies to a handful.

ETA: Each patient can have multiple diagnoses associated.

I'd like to generate some sort of temporal overview by year, so less technically minded people can appreciate the medical qualities of the dataset, but I haven't got a good way to do this other than a 250 by 20 table.

I was wondering if there was some interesting way of presenting this kind of data? (I.e. many sparse binary variables.)

• Are these diagnoses mutually exclusive, or is a patient 'able' (in your data that is) to have had multiple diagnoses during this 20 year period? – IWS Jun 30 '17 at 8:09
• @IWS patients can have multiple diagnoses associated. – Karl Damgaard Asmussen Jun 30 '17 at 8:19
• If you had only a few diagnoses (or diagnosis groups) I would recommend you to plot the proportion of patients with a certain disease per year. With time on the x-axis and the proportion our count on the y-axis. You could set the y-axis scale to a logarithmic one to accommodate the small and large groups in one graph. With 250 diagnoses however, I can see this becoming a mess... Is there no way to group these diagnoses? Or in other words: do you need to show all 250? – IWS Jun 30 '17 at 8:25
• @IWS there is a hierarchy to the diagnostic codes, but one of the goals is to also use the rarer and more specific things in a clinically significant manner. Perhaps I should make something interactive in an SVG or something... – Karl Damgaard Asmussen Jun 30 '17 at 8:28
• That would definitively be better. You could have the user start out with the top hierarchy, and provide options to zoom in on certain groups. – IWS Jun 30 '17 at 8:30

(If anyone wants to experiment with random data like this, my formula for count is Random Poisson( 40 / (diagnosis ^ 1.5) * (year + 30)), with diagnosis in 1..250 and year in 1..20.)