Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I ran two benchmark suites (X and Y) on two devices (D1 and D2). Device D1 complete the benchmarks after X1 and Y1 seconds and Device D2 in X2 and Y2 seconds.

I can very well calculateand& compare the relative performance in terms of speed of both the devices with-in the same benchmark X, e.g. if X1 (0.5) < X2 (2), then device D1 is four times faster than D2.

And I can do a similar interpretation for benchmark suite Y, e.g. if Y1 (2) < Y2 (3), then device D1 is 1.5 times faster than D2.

But a single device cannot be in two different speeds. I understand that devices will complete in different time based on benchmark suite. But does that means, should it yield various speed? Or Am I doing the speed calculation in right way?

I'm trying to arrive at a baseline to see how can I calculate a base speed? i.e the device D1 is always X times faster than Device D2, irrespective of benchmark suite.

share|improve this question
up vote 1 down vote accepted

I think the benchmarks measure different speeds because they probably use different functions of the machine. I would expect if the benchmarks are sufficiently complex that the differences in speed between the two machines are different for each benchmark. A possibility is to form the mean between the relative difference of the benchmarks, i.e. (X1/X2+Y1/Y2)/2, to get an approximation of how much faster one device is in comparison to another. This is of course, assuming that the quality of both benchmarks is comparable. One thorough benchmark is of course better then the mean of a good and a bad one.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.