# Providing variance measures for speedup ratios

I'm doing timings of an application which speeds up as we add threads. However, the times vary on repeated runs. What I'd like to have is a speedup chart which shows how many times faster than the baseline the application is, for each thread count I test on.

The data I was planning on gathering was 5-10 runs for each number of threads, from which I can get standard deviation for the times at each thread count. But I'm at a loss as to how to represent this deviation in terms of 1 std.dev error bars on a speedup-curve.

I'm thinking deviation I want to show is the std. of the speedup calcs, not the times intially right?

To make this more concrete:

• Suppose as a baseline, we have the times 10s,10s,10.1s,9.9s (sd = 0.081);
• at 4 threads, we have 2.5,2.5,8,2.8 (sd = 2.72);
• Speedups are 4,4,1.2625,3.5357 (sd = 1.3)

So the if I'm plotting the point for 4 threads, does it make sense to have the 1 std. error bar be 1.3 (2.6 in both dir)? ( ignoring skew of course. )

• I would be more concerned about the apparent large deviation from 2.5 to 8 for a single thread. Was this due to some other factor that isn't code dependent, e.g. memory dump or another program using the core at the same time? – adunaic Oct 10 '12 at 13:28
• I would do it on a log scale. – Mike Dunlavey Feb 7 '13 at 16:00