1
$\begingroup$

I am measuring the energy consumption and runtime of computer programs and would like to select a subset of the programs that have stable energy / runtimes. Each program is ran 10 times and I measure the energy consumption and runtime of each run. Runtime is measured in seconds and energy consumption in Joules.

I plan to apply a series of optimisations to the chosen subset and measure the change in energy consumption and runtime. I want to ensure that any measured change in energy or runtime is due to the applied optimisations and not just due to natural variation so I want to have a principled way of selecting programs with stable runtimes and energy measurements across the multiple runs.

I would like to know if there are any metrics that would allow me to say something on the lines of "select programs that have energy and run time variation of less than $x$%"?

Improve this question
$\endgroup$
4
  • 1
    $\begingroup$ My first two ideas are 1. standard deviation for the program, and 2. standard deviation for the program divided by the mean of the program. But I don't think you've really provided enough information about your data or your goal to get a really good answer. $\endgroup$
    – Michael Bishop
    Aug 3, 2018 at 14:37
  • $\begingroup$ @MichaelBishop I have added some more detail. $\endgroup$
    – David Greasley
    Aug 3, 2018 at 14:48
  • $\begingroup$ ok, I think standard error of the mean energy /runtime, and standard error divided by the mean would work. $\endgroup$
    – Michael Bishop
    Aug 3, 2018 at 19:15
  • $\begingroup$ SD divided by mean is coefficient to variation (CV): According to SD elephant sizes are more variable than ant sizes, according to CV ant sizes are more variable than elephant sizes. SD has measurement units as its units; CV has no units. Consider your situation before choosing. $\endgroup$
    – BruceET
    Aug 4, 2018 at 21:32

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Browse other questions tagged or ask your own question.