Boxplots vs. Confidence Intervals

I designed a heuristic that solves a problem concerning network graphs. It was tested on thousands of different instances that have various different characteristics: Topology, template, number and position of users, capacities, ... It produced more than 300000 results that also depend on the random seed that was used.

In order to evaluate the results, I decided to use boxplots that I created with JFreeChart. I created different diagrams for the different topologies and with separate plots for every template. I felt that was a good way to visually summarize the results.

I was asked, why I didn't use confidence intervals instead to give an estimate. From what I know, these depend on a underlying distribution of a population parameter while boxplots don't. However, I summarize results of different seeds, numbers of users and capacities. All these influence the results. So I think I would not be possible to use confidence intervals unless I distinguished every single network characteristic.

Is that true? What are other advantages and disadvantages? And how could I argue, that I only use boxplots and not confidence intervals?

• I see no "versus" here. Box plots show the entire distribution, summarized. You say you find them helpful. Confidence intervals arise when your concern is to estimate some parameter, say the mean of a variable, but quite possibly something else. In principle, the particular method you use to produce a confidence interval will depends on the underlying distribution, but with sample sizes $\sim 10^5$ that is not crucial. Nor does calculating a confidence interval for something depend on knowing everything in some sense: statistics would be impossible if so. Feb 13, 2015 at 10:44
• You have both or either depending on what you are trying to do; it is just not clear what your aims are here statistically. You sketch what you've done but I can't extract more from your post than that you want to evaluate results. Descriptive statistics, including box plots, might easily be enough, or only part of the answer. Feb 13, 2015 at 10:47
• I am evaluating the success rate, performance and quality of my heuristic. I also have the optimal results. I created boxplots for both to visually compare each of the three categories. I don't know if it makes sense to use confidence intervals here. Maybe to estimate the average runtime to calculate a result. I just don't really see the advantage of having a confidence interval for that compared to the boxplot. Feb 13, 2015 at 10:53
• Not trying to be flippant, but if you are uncertain about what you want statistically we can't tell you. With sample sizes such as you have, confidence intervals for the mean will be very narrow here, or so it would seem. I would be more concerned with the tails of the distribution and how bad results can be on occasion and the means won't tell you that directly (although clearly they will be affected by the extremes). (Detail: box plots don't usually show means, although many programs allow you to add means.) Feb 13, 2015 at 11:04
• I'd much prefer quantile plots. Feb 13, 2015 at 12:48