Error bars (or their likeness) for rankings

Question

This is a bit of a simple question I would suspect for many users here, but for someone like me who has never covered this in my statistics classes, this is posing a bit of a problem. I want to convert forecasted scores with a regular error bar into rankings with error bars.

For example, the following individuals are to be ranked.

A [Forecasted Score: 1000, Standard deviation: 25]

B [Forecasted Score: 940, Standard deviation: 35]

C [Forecasted Score: 950, Standard Deviation: 10]

D [Forecasted Score: 910, Standard deviation: 40]

The ranking without any way of expressing how uncertain you are would be the following.

1. A

2. C

3. B

4. D

Is this done using simply by rifling through all the combinations? As in, if I chose 2 standard deviations, A could be ranked 1, 2, or 3 while B could be ranked 1, 2, 3, or 4?

This stuff was never covered in my statistics classes, and the statistics books I have handy do not cover this. Any help would be greatly appreciated.

Edit:

I thought I would give a bit of background of what I am trying to do here. Suppose you are in a bar with your best friends and you are discussing a subject which can be ranked. Sports teams is popular but it can be decided clearly on the basis of who is winning and who is not. But judging songs? You're judging and ranking art. Taking a firm position on rankings might be helpful in saying that you have a ranking, but it doesn't help if someone else can make another convincing argument based on the same criteria why someone should be ranked over someone else. Sometimes it is just more helpful to describe the ranks it can have more than it does have.

This is the problem we are tackling here.

Answer 1

How about something like this?

Predictive rank probabilities
    individual
rank     A     B     C     D
   1 0.882 0.076 0.019 0.023
   2 0.097 0.292 0.507 0.104
   3 0.018 0.383 0.405 0.193
   4 0.003 0.248 0.069 0.680

This is a simple simulation-based approach based on treating the forecasted means and SDs as parameters of the predictive distribution of scores. Note that in this particular case, I made the (questionable!) assumptions that the forecasted distributions of scores for each individual are Normal and independent, just because there was no other information available in the question. So, anyway, we simply simulate 10,000 draws from this distribution, compute the ranks in each simulated run, and then summarize the results as a set of predicted probabilities of ranks. These can be interpreted either as (1) For a given rank, what are the probabilities that each individual will occupy that rank? or (2) For a given individual, what are the probabilities that they will occupy ranks 1-4?

R code follows:

# simulate draws from predictive distribution of scores
draws <- mapply(rnorm,
                mean = c(1000, 940, 950, 910),
                sd = c(25, 35, 10, 40),
                MoreArgs = list(n=10000))
# tabulate ranks in each simulation
ranks <- apply(draws, 1, order, decreasing=TRUE)
# computer predicted probability
probability <- sapply(1:4, function(x) rowMeans(ranks==x))
# line plot
par(mar=c(4,4,1.5,1.5))
matplot(y=probability, x=1:4, type="o", ylim=0:1, xlab="Rank",
        col="black", lty=1, pch=LETTERS[1:4])
# display probabilities
dimnames(probability) <- list(rank=1:4, individual=LETTERS[1:4])
round(probability, 3)

Answer 2

Ranking non-deterministic data in a deterministic way doesn't seem very sensible to me.

Let's take for example individual $A$. His forecasted score is $1000$ and $SD$ is $25$, but that doesn't mean that it couldn't be, e.g. $900$, $1100$ or even zero. It just means that these values are outside of the "expected range", heuristically speaking.

A more extreme example why the ranking system you proposed doesn't work very well is that it would place an imaginary individual with $0$ forecasted score and $SD$ of $2000$ on top.
In particular, you can't say "Individual $X$ will be ranked highest, because the upper bound of his expected score range is highest". You can only rank the forecast and say "Individual $X$ has the highest ranked forecasted score", because the actual outcome is random.

This of course assumes that you didn't mean "range" by "standard deviation", i.e. that scores for $A$ outside of $[975,1025]$ are impossible. I also assumed that the possible range for the actual score is the same for every individual, as the scoring system wouldn't make much sense otherwise.

Either way, to resolve this problem and get a meaningful ranking, you could just rank according to the "expected/forecasted values" and denote the ranking as such, involving the standard deviation values. There is no way to determine the ranking of the actual score results just by the use of forecasts.