# Statistically evaluating performance of algorithms

I am working on a problem for which I have come up with a couple of algorithms. To assess which one is the best, I compare a set of algorithm outputs (set of numbers) against the "true" set of values by evaluating the absolute value of the difference between the two. I then take the mean of this difference set and also calculate its standard deviation. My approach was to choose the algorithm with mean closest to zero as the best. The problem is that the standard deviation of the least mean algorithm is larger than some of the other algorithms. The relevant values are:

$$\begin{array}{|c|c|c|} \hline \mbox{Algorithm number}& \mbox{Mean} & \mbox{Standard Deviation} \\ \hline 1 & 0.316 & 0.615 \\ 2 & 0.298 & 0.615 \\ 3 & 0.253 & 0.657 \\ 4 & 0.283 & 0.657 \\ \hline \end{array}$$

Based on the mean, I would choose the third algorithm. Is this really the best algorithm, given the large standard deviation? And is there any other way to evaluate which algorithm is the best, just by studying the outputs?

• If you can elaborate a little more on what you are trying to do, it will greatly help us determine what "best" means in the context of your problem. "Best" is really contingent on what you are trying to do: without some understanding of that, we really won't be able to help you much. – David Marx Aug 5 '13 at 12:27
• @DavidMarx The algorithms are for determining one coordinate of a location using signal strength data. So the algorithms output this coordinate, which I compare against the known value as described. My thought was that the closer the difference of these values is to 0, the better the algorithm has worked. – Comp_Warrior Aug 5 '13 at 12:34