2
$\begingroup$

I'm not very familiar with statistical terms, so sorry if I messed up with some namings.

If I have a data set with names and ages.

Example:

Tom 12 Jenny 14 Joe 15 Kate 14

And I have a the following choices of ages: [11, 12, 13, 14, 15, 16]

Let's say I don't know the age of each person, I have a task to calculate the average error in age difference, if I randomly assign an age to a person by choosing one of the ages above.

So let's take an example, I randomly chose an age for each person:

Tom 13 Jenny 16 Joe 11 Kate 13

So for tom I have 1 year in difference from the real age, 2 for Jenny....

So the average error in age difference from the real age would be: (1 + 2 + 4 + 1)/4 = 2

I don't know how to think about it in a more general way. What can I use to get the average error in a large dataset if I chose random ages from a list?

Extra info: The purpose of this task is that, I am creating a machine learning algorithm that predict the age of each person based on other data, and I want to compare the results with the results of a random distribution of age. To see if my prediction gives better results (less average error) than just a random pick

Hope my question made sense.

Improve this question
$\endgroup$
1
  • $\begingroup$ Not really an answer: There's fairly neat result for RMS-difference, which will be an upper bound of the mean absolute difference (mean pairwise deviation). Or you should be able to use a triangle-inequality to get a bound. [In "nice" cases typically the mean deviation should tend to be about 80% of the RMS deviation.] $\endgroup$
    – Glen_b
    Commented Nov 18, 2017 at 16:39

1 Answer 1

Reset to default
1
$\begingroup$

Generalised version of your problem: Let $\mathcal{A} \equiv \{ a_1, ..., a_n \}$ be a multi-set of $n \in \mathbb{N}$ age values (which are non-negative real numbers, allowing for duplication), and suppose these ages are ordered as $a_1 \leqslant ... \leqslant a_n$. Consider two independent random variables $A, B \sim \text{IID U}(\mathcal{A})$ that are distributed uniformly on these ages; $A$ represents the true age of a person and $B$ represents your arbitrary prediction.

The prediction error of this prediction is $|A-B|$. The expected prediction error is:

$$\mathbb{E}(|A-B|) = \frac{1}{n^2} \sum_{i,j} |a_i-a_j| = \frac{2}{n^2} \sum_{i<j} (a_j - a_i).$$

This is the expected prediction error for a single guess of age for a single person, but if you take the average prediction error over a group of people, it will have the same expected value (by the linearity of expectations).

In the special case where you have ages $\mathcal{A} = \{ 11, 12, 13, 14, 15, 16 \}$ you obtain:

$$ \sum_{i<j} (a_j - a_i) = \begin{pmatrix} 1+2+3+4+5 \\ +1+2+3+4 \\ +1+2+3 \\ +1+2 \\ +1 \end{pmatrix} = (5 \cdot 1 + 4 \cdot 2 + 3 \cdot 3 + 2 \cdot 4 + 5) = 35,$$

which gives an expected prediction error of $\mathbb{E}(|A-B|) = 35 / 18 = 1.9 \dot{4}$. Again, this value is the expected prediction error for a single guess for a single person, but you will get the same expected value if you look at the average prediction error for a group of people.

Improve this answer
$\endgroup$
6
  • 1
    $\begingroup$ This is a good answer (+1) but I would like to suggest that a more appropriate solution for the intended application would account for the relative prevalence of each value within the dataset. Technically, then, the values should not be conceived of as a "set" but either as a sequence or a multiset. $\endgroup$
    – whuber ♦
    Commented Feb 22, 2018 at 0:43
  • 1
    $\begingroup$ Good point - I have updated the question to allow $\mathcal{A}$ to be a multi-set (i.e., to allow duplication of age values). The rest of the presentation effectively treats them like an ordered vector. Thanks for the suggestion. $\endgroup$
    – Ben
    Commented Feb 22, 2018 at 0:48
  • $\begingroup$ Note that the example given by the OP involves the multiset $\{12,14^2,15\}.$ Using the ages $11\ldots 16$ doesn't seem relevant (even though it was suggested by the OP). $\endgroup$
    – whuber ♦
    Commented Feb 22, 2018 at 0:52
  • $\begingroup$ That's the multi-set of true ages for the group of people, but the set $\{ 11, ..., 16 \}$ is the set of allowable ages for each individual person, which is used to make the prediction. (So he ends up making some predictions that are not in the multi-set of true ages.) The way I have framed the answer, it is for a single prediction for a single person, so I think for this purpose I am using the correct set. Does that sound right? $\endgroup$
    – Ben
    Commented Feb 22, 2018 at 1:05
  • $\begingroup$ The problem is that we do not, and cannot, know the distribution of ages produced by any particular prediction algorithm. The OP appears to want to compare a particular algorithm to a default procedure in which ages will be "randomly" guessed. A naive guessing algorithm would simply guess exactly the values appearing in the dataset, independently with probabilities proportional to their frequencies. (This is a model-free, parameter-free bootstrap.) Any other algorithm would implicitly be modeling the data in some way and therefore would not really qualify as a reference for comparison. $\endgroup$
    – whuber ♦
    Commented Feb 22, 2018 at 14:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.