# Visualize sets and their connections

I collected a data-sets which tested for many users (>100.000) which out of ten features in a software-product they use. They can use multiple features but for each feature there is only "use" or "don't use". So in terms of software-development, I have many sets of features in a length of zero to ten.

I want to visualize the data in such a way, that it answers two questions:

1. Which features of the software are the most popular ones; How many percent of the users use FeatureX?
2. Which features are most often used together? So which sets of features are the most popular. For example, if most people tend to use Feature1 together with Feature2 (no matter what other features they use or don't use), I want to see this somehow in my visualization.

To clarify: The data is already collected, I'm searching for a good way to show this data.

I'm not sure whether you can (and should) visualize this in one graph or whether you should choose to display multiple graphs.

I came up with a simple solution for 1.: Creating a bar-graph which shows for each feature how many people selected it solves this problem. But it doesn't help for 2.

Let $C$ by a table and $C[i,j]$ is number of users that use both $i$-th and $j$-th feature. $C[i,i]$ is number of users using $i$-th feature. By $N$ we denote the total number of users.

One possibility is to plot the table, and (below there are some suggestions):

• only its left triangular part (as $C[i,j]=C[j,i]$),
• with entries sorted in a way where frequently co-used features are together,
• with coloring/brightness more-or-less proportional to $\log C[i,j]$.

Other possibility is to construct a graph out of your data. Nodes are features and the edges connecting them - the indication of co-usage.

To obtain it you can compute relative co-usage $$c[i,j]= \frac{C[i,j]}{\sqrt{C[i,i] C[j,j]}}$$

• For not correlated features it is $$c_{non-corr}[i,j]= \frac{N \frac{C[i,i]}{N} \frac{C[j,j]}{N}}{\sqrt{C[i,i] C[j,j]}} = \frac{\sqrt{C[i,i] C[j,j]}}{N}.$$
• If it is much higher (up to $1$) then the features are co-used.
• If it is much lower (down to $0$) - the features are anti-correlated (i.e. people tend to use either $i$ or $j$), which may be a common phenomena as well.

You need to set thresholds $t_c$ (and optionally $t_a<t_c$ ):

• If $\frac{c[i,j]}{c_{non-corr}[i,j]}>t_c$ connect $i$ and $j$, to mark them as the co-used features.
• Optionally, if $\frac{c[i,j]}{c_{non-corr}[i,j]}<t_a$ connect $i$ and $j$ with a different type of lines, to mark them as anti-correlated.

Displaying the actual numbers on the plot (or point/line sizes) may be useful.

EDIT: Fixed an error.

• +1 Good response. I think Cytoscape might be helpful for the 2nd part of your response. (It was used in a recent paper, Comorbidity: A network perspective, BBS (2010) 33:137, in a spirit similar to the one you described here.) – chl Oct 30 '11 at 16:50

I think you may be interested in circular displays for tabular data (in your case, a two-way table denoting the co-occurence of every binary features), as proposed through Circos; see example and on-line demo here.

Sidenote: As an alternative, you can also take a look at Parallel Sets that were developed by Robert Kosara. See also,

Robert Kosara, Turning a Table into a Tree: Growing Parallel Sets into a Purposeful Project, in Steele, Iliinsky (eds), Beautiful Visualization, pp. 193–204, O'Reilly Media, 2010.

The simple approach for #2 would be a cross-tabulation: the list of 10 features across the top and down the side, with the intersecting usage of each feature shown as a count, or as various percentages. Percentages are incredibly flexible: you can base the percentages to column count, or table count, and those counts can be unique users or unique user-feature pairs. For that reason I'd start with the counts. Counts are nice and simple and close to the original data. But, your original question sounds like you'll need to do a percentage at some point.

Using conditional formatting in Excel is a quick and dirty way to then see the relative magnitude - set it to a color scale.

A couple easy ideas:

• bar graph of pairs (or sets) of features
• a separate bar graph, for each feature, of how many times each other feature appears with it
• Perhaps my question was not to clear: I don't want to reduce the analysis to only the pairs(!) of features which occur together very often but also bigger sets. Anyway, thanks for your answer! – theomega Oct 30 '11 at 15:40
• Or sets, if you wish. My real point was to try the obvious non-fancy things before spending a lot of time on fancier visualizations... – glenn mcdonald Oct 31 '11 at 1:28