# Estimating the average error of a random distribution

I'm not very familiar with statistical terms, so sorry if I messed up with some namings.

If I have a data set with names and ages.

Example:

Tom 12 Jenny 14 Joe 15 Kate 14

And I have a the following choices of ages: [11, 12, 13, 14, 15, 16]

Let's say I don't know the age of each person, I have a task to calculate the average error in age difference, if I randomly assign an age to a person by choosing one of the ages above.

So let's take an example, I randomly chose an age for each person:

Tom 13 Jenny 16 Joe 11 Kate 13

So for tom I have 1 year in difference from the real age, 2 for Jenny....

So the average error in age difference from the real age would be: (1 + 2 + 4 + 1)/4 = 2

I don't know how to think about it in a more general way. What can I use to get the average error in a large dataset if I chose random ages from a list?

Extra info: The purpose of this task is that, I am creating a machine learning algorithm that predict the age of each person based on other data, and I want to compare the results with the results of a random distribution of age. To see if my prediction gives better results (less average error) than just a random pick

• Not really an answer: There's fairly neat result for RMS-difference, which will be an upper bound of the mean absolute difference (mean pairwise deviation). Or you should be able to use a triangle-inequality to get a bound. [In "nice" cases typically the mean deviation should tend to be about 80% of the RMS deviation.] – Glen_b -Reinstate Monica Nov 18 '17 at 16:39

Generalised version of your problem: Let $\mathcal{A} \equiv \{ a_1, ..., a_n \}$ be a multi-set of $n \in \mathbb{N}$ age values (which are non-negative real numbers, allowing for duplication), and suppose these ages are ordered as $a_1 \leqslant ... \leqslant a_n$. Consider two independent random variables $A, B \sim \text{IID U}(\mathcal{A})$ that are distributed uniformly on these ages; $A$ represents the true age of a person and $B$ represents your arbitrary prediction.

The prediction error of this prediction is $|A-B|$. The expected prediction error is:

$$\mathbb{E}(|A-B|) = \frac{1}{n^2} \sum_{i,j} |a_i-a_j| = \frac{2}{n^2} \sum_{i<j} (a_j - a_i).$$

This is the expected prediction error for a single guess of age for a single person, but if you take the average prediction error over a group of people, it will have the same expected value (by the linearity of expectations).

In the special case where you have ages $\mathcal{A} = \{ 11, 12, 13, 14, 15, 16 \}$ you obtain:

$$\sum_{i<j} (a_j - a_i) = \begin{pmatrix} 1+2+3+4+5 \\ +1+2+3+4 \\ +1+2+3 \\ +1+2 \\ +1 \end{pmatrix} = (5 \cdot 1 + 4 \cdot 2 + 3 \cdot 3 + 2 \cdot 4 + 5) = 35,$$

which gives an expected prediction error of $\mathbb{E}(|A-B|) = 35 / 18 = 1.9 \dot{4}$. Again, this value is the expected prediction error for a single guess for a single person, but you will get the same expected value if you look at the average prediction error for a group of people.

• This is a good answer (+1) but I would like to suggest that a more appropriate solution for the intended application would account for the relative prevalence of each value within the dataset. Technically, then, the values should not be conceived of as a "set" but either as a sequence or a multiset. – whuber Feb 22 '18 at 0:43
• Good point - I have updated the question to allow $\mathcal{A}$ to be a multi-set (i.e., to allow duplication of age values). The rest of the presentation effectively treats them like an ordered vector. Thanks for the suggestion. – Ben - Reinstate Monica Feb 22 '18 at 0:48
• Note that the example given by the OP involves the multiset $\{12,14^2,15\}.$ Using the ages $11\ldots 16$ doesn't seem relevant (even though it was suggested by the OP). – whuber Feb 22 '18 at 0:52
• That's the multi-set of true ages for the group of people, but the set $\{ 11, ..., 16 \}$ is the set of allowable ages for each individual person, which is used to make the prediction. (So he ends up making some predictions that are not in the multi-set of true ages.) The way I have framed the answer, it is for a single prediction for a single person, so I think for this purpose I am using the correct set. Does that sound right? – Ben - Reinstate Monica Feb 22 '18 at 1:05
• The problem is that we do not, and cannot, know the distribution of ages produced by any particular prediction algorithm. The OP appears to want to compare a particular algorithm to a default procedure in which ages will be "randomly" guessed. A naive guessing algorithm would simply guess exactly the values appearing in the dataset, independently with probabilities proportional to their frequencies. (This is a model-free, parameter-free bootstrap.) Any other algorithm would implicitly be modeling the data in some way and therefore would not really qualify as a reference for comparison. – whuber Feb 22 '18 at 14:39