Given a set of data points, if a single point which is above/below the average is excluded, can it be said that the new average will surely decrease/increase? What if the average is the median instead of the mean?
2 Answers
For the arithmetic mean, we can base a proof on the update formula, where $\bar{x}_n$ is the arithmetic mean based on $n$ observations. Then we can see: $$ \bar{x}_{n+1}= \frac{x_1+\dotsc+x_n+x_{n+1}}{n+1}= \frac{n}{n+1}\cdot \bar{x}_n + \frac1{n+1}\cdot x_{n+1} $$ By substituting $n+1=m$ we find the downdate formula: $$ \bar{x}_{m-1}= \frac{m}{m-1}\cdot \left\{ \bar{x}_m -\frac1{m}\cdot x_m\right\} = \bar{x}_m -\frac1{m-1}(x_m-\bar{x}_m) $$ and now the conclusion is pretty obvious. For other means which can be presented as (monotone functions of) some linear form (of transformed data), like geometric or harmonic mean, can be proved similarly.
But, the median is not of this form, since it is possible to perturb some data without changing the median at all. So an argument must be framed otherwise. Assume that $n=2m+1$ is odd, and change indices so data is in increasing order. Then the median is the middle term $x_{m+1}$. Exclude one of the points less than the median, say $x_s$ with $s\le m$. Then we have an even number of points, and the new ordered data is (maintaining original indices, only leaving out $s$): $$ x_1 \le x_2 \le \dotsm x_{s-1}\le x_{s+1} \le \dotsm $$ Now the new median is $\frac{x_{m+1}+x_{m+2}}{2}$, which cannot be smaller than the old median. But it can be equal. Other cases treated likewise.
This really depends on how the measure you are interested in is formed. The statement is obviously true for the arithmetic mean, as well as the geometric and harmonic mean, of a set of reals. For the median (as well as the mode), you can easily construct a counter-example, e.g. (1, 2, 2, 2, 3).