0
$\begingroup$

I am teaching some very basic statistics for grouped data.

In a different class a few years ago I made an 'assumption of uniformity' about the data:

within bins, data is uniformly distributed

E.g. if I had three data points in the $[0,1)$ bin, under this assumption, the data points would be $0.25$, $0.5$ and $0.75$.

I understand that this isn't a very good assumption but is certainly a working one. Previously, I used this assumption to give the mode as the midpoint of the modal class and I could also find the median in the following way:

Suppose we have 0-1 with two entries, 1-2 with three and 2-3 with one. There are six data points so the median will be at position 3.5 at the midpoint of the two middle data, points, so at 1.375 according to the assumption of uniformity.

Since that time I have learnt that we can use the ogive to give a more reasonable approximation for the median (without this assumption) and I also seen that there is a formula that gives an approximation to the mode without using this assumption.

Just recently I have seen a formula for the approximate location of the median that doesn't use this assumption either.

This is all great and I am quite happy to show my students all of this. However, it is under this assumption of uniformity that the formula for the mean of a grouped data set is correct! I find this quite an inconsistency and I am not comfortable using the assumption for the mean and not for the median and mode.

I normally like deriving all the formulae I use in an appendix: with time pressure this is not possible...

My questions therefore are:

  1. Is there a well-known formula for the mean which doesn't use this assumption of uniformity?
  2. If not, can anyone account for this apparent anomaly? Or is the usual formula a better estimator than I give it credit for and can be derived without using an assumption of uniformity?

I don't see how overestimations can necessarily be compensated with underestimations. Surely assuming that all the data is centred at the midpoint is an even stronger assumption.

$\endgroup$
  • $\begingroup$ with three points, I'd call data points at 0.25, 0.5 and 0.75 something like "evenly spaced" rather than "uniformly distributed". $\endgroup$ – Glen_b -Reinstate Monica Jan 27 '16 at 16:44
  • $\begingroup$ Are you "teaching" or "taught"? $\endgroup$ – Xi'an Jan 27 '16 at 19:43
1
$\begingroup$

All such formulae must make assumptions to achieve what they achieve and the common assumption is uniformity. They just may not make the assumption explicit.

I like the ogive plot but the common plot of connected lines implicitly assumes uniformity since that is required to "connect-the-dots" between actual data...justifies the linear interpolation.

A more accurate plot is the empirical distribution plot which "steps" at each bin. But the EDF plot can be made to step at the start of the bin, at the end of the bin, in the middle of the bin, or anywhere else within the bin. The ogive plot is what you get from the EDF when you connect the dots rather than stepping at the dots.

In the example you give: {[0-1)x2, [1-2)x3, [2-3]x1}, you give the median as 1.375 assuming uniformity. Without an underlying justification that these are binned data, the classical median is the second bin: literally any point within the second bin: 1 is as good as 1.375 is as good as 1.5 is as good as 1.99.

For that matter, this is true even without the binning. If we had data {1, 1, 1, 2, 2, 2}, people report the median as 1.5. But the actual median is any value between 1 and 2, inclusive. 1.5 is no better than 1.2 or 1.8 or 1 or 2. In fact, 1.5 is an absurd representative value if 1 and 2 are the only possible values and it does not have the "fair-share-value" excuse that the mean has.

So any "algorithm" to get a "better" estimate of the median must rely on some assumption about how the points scatter on a more fine level than the bin. And this makes them subject to error when this assumption is wrong.

The general notion of uniformity is that at the binning level, the distribution is hopefully not changing very much within the bin (otherwise, binning was a VERY bad idea).

$\endgroup$
  • $\begingroup$ I did not know that the median and mode formulae were also derived under this assumption. Thank you very much. As it happens I would try not to draw the ogive as piecewise linear. $\endgroup$ – JP McCarthy Jan 27 '16 at 14:34
  • $\begingroup$ ...this really shows the value of deriving all of these formulae... $\endgroup$ – JP McCarthy Jan 27 '16 at 14:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.