The following chart compares two types of computational workloads based on six aspects of their interaction with the memory system. While it is obvious that individual benchmarks should be grouped according to workload type, it is less obvious how they should be ordered within a type. (I suspect that placing the SPEC CPU 2006 benchmarks at the top and the server benchmarks at the bottom is better because SPEC CPU is the default processor benchmark and the desire is to justify a different memory system for server workloads.)

SPEC CPU and Server front-end and back-end memory system overhead

For convenience, I ordered the SPEC CPU 2006 benchmarks by decreasing back-end memory system execution time overhead and the server benchmarks by increasing front-end memory system overhead. This ordering does provide some aesthetic appeal and limits variation to one broad set of values (total back-end for server, total front-end for SPEC), but it may emphasize false similarity (e.g., 'lbm' is in a sense more like 'mil' than its neighbors 'cal' and 'omn' which have large L3 cache components). On the other hand, the regularity in totals may provide an order that actually facilitates using the colors to make comparisons of components.

I suspect using a more sophisticated measure of workstation-like (SPEC CPU) versus server-like nature might make the differences clearer.

I thought that perhaps the difference in distances between an item and the two category averages might be good (using the six values to form a six-dimensional space), but there might be a better method of ordering.

In addition if a ranking is used, there is the question of whether items in one category that are more similar to the other category should be placed nearer the other category or farther away from the meeting point. I.e., should the items be ordered so that if the ranking were plotted it would form a trough, a hill, a descent to a cliff face, or an ascent to a cliff edge?

The above chart was an attempt to improve a chart from a computer science paper (Figure 2 in Aamer Jaleel et al., "High Performing Cache Hierarchies for Server Workloads: Relaxing Inclusion to Capture the Latency Benefits of Exclusive Caches", 2015 (PDF)). I think it is a noticeable improvement over the original:

Modified Figure 2 from "High Perfoming Cache Hierarchies for Server Workloads"

(In addition to cutting out the "With Prefetching" section, the above modified image transferred the legend from that section to this section.)


What is the value of showing the individual bars? If it's to look-up a particular benchmark by name, then alphabetical may be best. If it's just show a variety of profiles, any approach seems fine and yours is at least aesthetic. Ordering both groups by total (BE+FE) memory time would be more "regular" in some sense.

Regardless, one change to consider is flipping the emphasis of the summary and the individual tests. You say the take-away is that server tasks have different memory demands, so why not make the summaries prominent and the individual benchmarks faded?

If you're concerned about keeping certain similar benchmarks together, you could categorize them, such as by the largest component (all those where Memory is highest together, etc.). It's coarse, but at least cal and omn would be together instead of next to lbm.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.