Comparison of two values in a statistically sound manner For a university paper, I have to compare whether two program versions are statistically significantly different. The comparison in terms of runtime performance is straightforward -- I determine whether the confidence intervals for the two sets of measurements overlap. However, in the case of memory footprint, I have no sets of measurements, because the memory footprint does not vary from measurement to measurement (e.g., version 1 has a constant footprint of 920 kb and version 2 consumes 1040 kb). Is there any way to compare both values in a statistically sound manner?
 A: It is quite simple; memory footprint is constant, this means it has no deviation and thus any difference is significant. 
A: You're incorrect about the confidence intervals.  Assuming they are 95% confidence intervals then they can overlap quite a bit and be statistically significantly different.  What you need is for the confidence interval of the difference between the two to not overlap 0.  The standard error of the difference is sqrt(2 *((var1+var2)/(2(n-1)))/n) and the df for the confidence interval is 2(n-1) (assuming you ran the test equal numbers of times for both programs).
Alternatively you could average the w (one side of the confidence interval) from each condition and then multiply that by sqrt(2).  That will give you a minimum difference for statistical significance.
If the program can run quickly just run it 1e5 times or more (lots and lots) and call the resulting means and standard deviations approximate population values.  Your standard error will be so small that a confidence interval will be almost 0 and any difference would be significant.  You could even report overlayed histogram plots and CDFs.
(while I would endorse this last method you might need to do the CIs anyway)
