Comparing the results of two different benchmark results?

Question

I ran two benchmark suites (X & Y) on two devices (D1 & D2). Device D1 complete benchmark after X1,Y1 seconds and Device D2 in X2,Y2 seconds.

I can very well calculate & compare the relative performance in terms of speed of both the devices with-in the same benchmark X. eg: if X1 ( 0.5 ) < X2 (2), then Device D1 is 4 times faster than D2.

And I can do a similar interpretation for benchmark suite Y . eg: if Y1 (2) < Y2 (3), then Device D1 is 1.5 times faster than D2.

But a single device cannot be in two different speeds. I understand that devices will complete in different time based on benchmark suite. But does that means, should it yeild various speed? Or Am I doing the speed calculation in right way?

I'm trying to arrive at a baseline to see how can I calculate a base speed? i.e the device D1 is always X times faster than Device D2, irrespective of benchmark suite.

Answer 1

I think the benchmarks measure different speeds because they probably use different functions of the machine. I would expect if the benchmarks are sufficiently complex that the differences in speed between the two machines are different for each benchmark. A possibility is to form the mean between the relative difference of the benchmarks, i.e. (X1/X2+Y1/Y2)/2, to get an approximation of how much faster one device is in comparison to another. This is of course, assuming that the quality of both benchmarks is comparable. One thorough benchmark is of course better then the mean of a good and a bad one.