At many question-and-answer sites, like StackExchange, people can upvote or downvote each answer. These sites also typically try to use the votes to sort answers, so the answers that are most likely to be helpful or accurate tend to appear nearest the top. Given the number of upvotes and downvotes for each answer, how should one sort the answers?
Examples. To illustrate why I think this question might have some non-trivial statistical content, let me survey a handful of methods one might consider and some shortcomings of them. In each method, we compute a score for each answer from its upvotes and downvotes, and then sort the answers by score, so the only question is what method to use to compute the score.
Upvotes minus downvotes. The additive difference is simple to compute. Limitations. Is an answer with 121 upvotes and 100 downvotes really better than an answer with 20 upvotes and 0 downvotes?
Upvotes divided by total number of votes. This estimates the fraction of voters who held a positive opinion of the answer, which is arguably natural and meaningful in its own right. Limitations. Is an answer with 1 upvote and 0 downvotes really better than an answer with 8 upvotes and 1 downvotes?
Compute a confidence interval. I suppose we could compute, for each answer, a confidence interval for the true fraction of people who would vote positively if they voted. But it is not clear how to extend this to a complete sorting scheme; what do we do when two answers have overlapping confidence intervals?
Laplace smoothing. We could apply Laplace smoothing (additive smoothing) to the counts of upvotes and downvotes, then estimate the fraction of votes that are positive. If we have $u$ upvotes and $d$ downvotes for an answer, its score will be $(u+1)/(u+d+2)$. (For instance, this would declare that an answer with 3 upvotes and 1 downvote is equivalent an answer with 1 upvote and 0 downvotes. Does that seem reasonable? Hard to say.)